Solid-state image pickup element, method of controlling a solid-state  image pickup element, and image pickup apparatus

ABSTRACT

A solid-state image pickup element, including: a pixel array including a plurality of pixels; a first calculator that calculates a phase difference evaluation value for focus detection by a phase difference detection method based on signal from the pixel; and a second calculator that calculates a contrast evaluation value for focus detection by a contrast detection method based on signal from the pixel, wherein, when the first calculator completes calculation of the phase difference evaluation value, the phase difference evaluation value is output regardless of whether or not output of an image signal acquired by the pixel array is completed, and wherein, when the second calculator completes calculation of the contrast evaluation value, the contrast evaluation value is output regardless of whether or not output of the image signal acquired by the pixel array is completed.

BACKGROUND OF THE INVENTION Field of the Invention

This disclosure relates to a solid-state image pickup element, a methodof controlling a solid-state image pickup element, and an image pickupapparatus.

Description of the Related Art

Image pickup apparatus such as a digital camera and a video camera areequipped with an autofocus (AF) function of automatically focusing on anobject (automatically adjusting a focus position of a lens).

In order to achieve a fast autofocus, signals for focus detection arerequired to be read out at a high frame rate. However, when the signalsfor focus detection are read out at a high frame rate, increase inamount of data to be transferred to a signal processor located at asubsequent stage of a solid-state image pickup element is caused. Inview of this, the following solid-state image pickup element has beenproposed. The solid-state image pickup element includes an AF evaluationvalue detection unit configured to detect an AF evaluation value, and isconfigured to output the AF evaluation value detected by the AFevaluation value detection unit to the outside (see Japanese PatentApplication Laid-Open No. 2015-12489).

Incidentally, as an AF method, there has been known a hybrid AF, whichuses a combination of an AF by a phase difference detection method, thatis, a phase difference AF, and an AF by a contrast detection method,that is, a contrast AF. The hybrid AF generally employs a method ofmoving a focus lens to a position near an in-focus position by the phasedifference AF, and further moving the focus lens to the in-focusposition by the contrast AF. With this method, a time required forobtaining an in-focus state can be reduced, and an in-focus accuracy canbe improved. However, in Japanese Patent Application Laid-Open No.2015-12489, no consideration is given to the hybrid AF technology, andhence a fast and highly accurate autofocus has not always been achieveddepending on image pickup conditions and the like.

SUMMARY OF THE INVENTION

According to an aspect of an embodiment, there is provided a solid-stateimage pickup element, including: a pixel array including a plurality ofpixels; a first calculator that calculates a phase difference evaluationvalue for focus detection by a phase difference detection method basedon signal from the pixel; and a second calculator that calculates acontrast evaluation value for focus detection by a contrast detectionmethod based on signal from the pixel, wherein, when the firstcalculator completes calculation of the phase difference evaluationvalue, the phase difference evaluation value is output regardless ofwhether or not output of an image signal acquired by the pixel array iscompleted, and wherein, when the second calculator completes calculationof the contrast evaluation value, the contrast evaluation value isoutput regardless of whether or not output of the image signal acquiredby the pixel array is completed.

According to another aspect of an embodiment, there is provided a methodof controlling a solid-state image pickup element, including: outputtingan image signal acquired by a pixel array including a plurality ofpixels; calculating a phase difference evaluation value for focusdetection by a phase difference detection method based on signal fromthe pixel; calculating a contrast evaluation value for focus detectionby a contrast detection method based on signal from the pixel;outputting, when calculation of the phase difference evaluation value iscompleted, the phase difference evaluation value regardless of whetheror not output of the image signal acquired by the pixel array iscompleted; and outputting, when calculation of the contrast evaluationvalue is completed, the contrast evaluation value regardless of whetheror not output of the image signal acquired by the pixel array iscompleted.

According to further another aspect of an embodiment, there is providedan image pickup apparatus, including: a solid-state image pickupelement, including: a pixel array including a plurality of pixels; afirst calculator that calculates a phase difference evaluation value forfocus detection by a phase difference detection method based on signalfrom the pixel; and a second calculator that calculates a contrastevaluation value for focus detection by a contrast detection methodbased on signal from the pixel, the solid-state image pickup elementbeing configured to: output, when the first calculator completescalculation of the phase difference evaluation value, the phasedifference evaluation value regardless of whether or not output of animage signal acquired by the pixel array is completed; and output, whenthe second calculator completes calculation of the contrast evaluationvalue, the contrast evaluation value regardless of whether or not outputof the image signal acquired by the pixel array is completed; and acontroller that performs control for driving a focus lens based on thephase difference evaluation value and the contrast evaluation value.

According to further another aspect of an embodiment, there is provideda solid-state image pickup element, including: a pixel array including aplurality of pixels arranged in matrix; a first calculator thatcalculates a phase difference evaluation value for focus detection by aphase difference detection method based on a signal from a pixel forphase difference detection included in the plurality of pixels; aninterpolation processor that generates a signal for compensating for adefect, which is caused in an image signal acquired by the pixel arrayand caused because the pixel for phase difference detection is includedin the plurality of pixels, through interpolation using a signal from apixel other than the pixel for phase difference detection; and a secondcalculator that calculates a contrast evaluation value for focusdetection by a contrast detection method based on the image signalincluding the signal generated by the interpolation processor throughthe interpolation.

According to further another aspect of an embodiment, there is provideda method of controlling a solid-state image pickup element, including:calculating a phase difference evaluation value for focus detection by aphase difference detection method based on a signal from a pixel forphase difference detection included in a plurality of pixels arranged inmatrix in a pixel array; generating a signal for compensating for adefect, which is caused in an image signal acquired by the pixel arrayand caused because the pixel for phase difference detection is includedin the plurality of pixels, through interpolation using a signal from apixel other than the pixel for phase difference detection; andcalculating a contrast evaluation value for focus detection by acontrast detection method based on the image signal including the signalgenerated through the interpolation.

According to further another aspect of an embodiment, there is providedan image pickup apparatus, including: a solid-state image pickup elementincluding: a pixel array including a plurality of pixels arranged inmatrix; a first calculator that calculates a phase difference evaluationvalue for focus detection by a phase difference detection method basedon a signal from a pixel for phase difference detection included in theplurality of pixels; an interpolation processor that generates a signalfor compensating for a defect, which is caused in an image signalacquired by the pixel array and caused because the pixel for phasedifference detection is included in the plurality of pixels, throughinterpolation using a signal from a pixel other than the pixel for phasedifference detection; and a second calculator that calculates a contrastevaluation value for focus detection by a contrast detection methodbased on the image signal including the signal generated by theinterpolation processor through the interpolation; and a controller thatperforms control for driving a focus lens based on the phase differenceevaluation value and the contrast evaluation value.

According to further another aspect of an embodiment, there is provideda solid-state image pickup element, including: a plurality of imagepickup pixels that each receive light condensed by an image pickupoptical system; a plurality of phase difference detection pixels thateach receive a pair of light fluxes passing through different pupilpartial regions of the image pickup optical system; a circuit that readsout image pickup signal from the image pickup pixel, and to read outphase difference signal from the phase difference detection pixel; acontrast evaluation value calculator that calculates a contrastevaluation value based on the image pickup signal; a phase differenceevaluation value calculator that calculates a phase differenceevaluation value based on the phase difference signal before readout ofthe image pickup signal is completed; and a determination unit thatdetermines whether or not to calculate the contrast evaluation valuebased on the phase difference evaluation value.

According to further another aspect of an embodiment, there is providedan image pickup apparatus, including: a solid-state image pickup elementincluding: a plurality of image pickup pixels that each receive lightcondensed by an image pickup optical system; a plurality of phasedifference detection pixels that each receive a pair of light fluxespassing through different pupil partial regions of the image pickupoptical system; a circuit that reads out image pickup signal from theimage pickup pixel, and to read out phase difference signal from thephase difference detection pixel; a contrast evaluation value calculatorthat calculates a contrast evaluation value based on the image pickupsignal; a phase difference evaluation value calculator that calculates aphase difference evaluation value based on the phase difference signalbefore readout of the image pickup signal is completed; and adetermination unit that determines whether or not to calculate thecontrast evaluation value based on the phase difference evaluationvalue; and a controller that performs an in-focus operation of the imagepickup optical system based on any one of the phase differenceevaluation value and the contrast evaluation value.

According to further another aspect of an embodiment, there is provideda method of controlling a solid-state image pickup element, thesolid-state image pickup element including: a plurality of image pickuppixels that each receive light condensed by an image pickup opticalsystem; a plurality of phase difference detection pixels that eachreceive a pair of light fluxes passing through different pupil partialregions of the image pickup optical system; and a circuit that reads outimage pickup signal from the image pickup pixel, and to read out phasedifference signal from the phase difference detection pixel, the methodincluding: calculating a phase difference evaluation value based on thephase difference signal before readout of the image pickup signal iscompleted; and determining whether or not to calculate a contrastevaluation value based on the image pickup signal, depending on thephase difference evaluation value.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram for illustrating a configuration of an imagepickup apparatus according to a first embodiment.

FIG. 2A is a perspective view for illustrating a structure of asolid-state image pickup element according to the first embodiment.

FIG. 2B and FIG. 2C are circuit diagrams for illustrating thesolid-state image pickup element according to the first embodiment.

FIG. 3A and FIG. 3B are diagrams for illustrating pixels arranged in anormal row and an image pickup optical system.

FIG. 4A and FIG. 4B are diagrams for illustrating pixels arranged in aphase difference AF row and the image pickup optical system.

FIG. 5 is a diagram for illustrating an example of rows selected at thetime of readout in the solid-state image pickup element according to thefirst embodiment.

FIG. 6 is a time chart for illustrating an operation of the image pickupapparatus according to the first embodiment.

FIG. 7 is a flow chart for illustrating the operation of the imagepickup apparatus according to the first embodiment.

FIG. 8 is a time chart for illustrating an operation of an image pickupapparatus according to a second embodiment.

FIG. 9 is a flow chart for illustrating the operation of the imagepickup apparatus according to the second embodiment.

FIG. 10 is a circuit diagram for illustrating a solid-state image pickupelement according to a third embodiment.

FIG. 11A and FIG. 11B are diagrams for conceptually illustrating alayout of a pixel array and signals to be output.

FIG. 11C is a time chart for illustrating an example of an operation ofan image pickup apparatus according to the third embodiment.

FIG. 11D is a time chart for illustrating another example of theoperation of the image pickup apparatus according to the thirdembodiment.

FIG. 12A is a diagram for conceptually illustrating signals to be outputfrom a solid-state image pickup element according to Reference Example1.

FIG. 12B is a time chart for illustrating an example of an operation ofan image pickup apparatus according to Reference Example 1.

FIG. 12C is a time chart for illustrating another example of theoperation of the image pickup apparatus according to Reference Example1.

FIG. 13 is a circuit diagram for illustrating a solid-state image pickupelement according to a fourth embodiment.

FIG. 14 is a diagram for illustrating a layout of pixels in thesolid-state image pickup element according to the fourth embodiment.

FIG. 15A and FIG. 15B are diagrams for conceptually illustratinginterpolation processing.

FIG. 16 is a time chart for illustrating an operation of an image pickupapparatus according to the fourth embodiment.

FIG. 17 is a flow chart for illustrating the operation of the imagepickup apparatus according to the fourth embodiment.

FIG. 18 is a time chart for illustrating an operation of an image pickupapparatus according to a fifth embodiment.

FIG. 19 is a flow chart for illustrating the operation of the imagepickup apparatus according to the fifth embodiment.

FIG. 20 is a schematic diagram for illustrating a configuration of asolid-state image pickup element according to a sixth embodiment.

FIG. 21 is a schematic diagram for illustrating a configuration of apixel in the image pickup element according to the sixth embodiment.

FIG. 22A and FIG. 22B are schematic diagrams for illustrating astructure of image pickup pixels of the image pickup element accordingto the sixth embodiment.

FIG. 23A and FIG. 23B are schematic diagrams for illustrating astructure of phase difference detection pixels in the image pickupelement according to the sixth embodiment.

FIG. 24 is a schematic diagram for illustrating a focus detection regionon a pixel array in the image pickup element according to the sixthembodiment.

FIG. 25A, FIG. 25B, and FIG. 25C are schematic graphs for showing anexample of a pair of phase difference signals for focus detection in theimage pickup element according to the sixth embodiment.

FIG. 26A and FIG. 26B are graphs for showing a relationship between ashift amount and a correlation amount of the phase difference signals inthe image pickup element according to the sixth embodiment.

FIG. 27A and FIG. 27B are graphs for showing a relationship between theshift amount and a correlation change amount of the phase differencesignals in the image pickup element according to the sixth embodiment.

FIG. 28 is a schematic diagram for illustrating an arrangement ofsub-pixels in the image pickup element according to the sixthembodiment.

FIG. 29 is a timing chart for illustrating a method of controlling animage pickup element according to the sixth embodiment.

FIG. 30 is a flow chart for illustrating the method of controlling animage pickup element according to the sixth embodiment.

FIG. 31 is a schematic diagram for illustrating an image pickup elementconfiguration in an image pickup element according to a seventhembodiment.

FIG. 32 is a timing chart for illustrating a method of controlling animage pickup element according to the seventh embodiment.

FIG. 33 is a flow chart for illustrating the method of controlling animage pickup element according to the seventh embodiment.

FIG. 34 is a timing chart for illustrating an example of a hybrid AF inan image pickup apparatus according to Reference Example 2.

DESCRIPTION OF THE EMBODIMENTS

Preferred embodiments of the present invention will now be described indetail in accordance with the accompanying drawings.

First Embodiment

A solid-state image pickup element, a method of controlling asolid-state image pickup element, and an image pickup apparatusaccording to a first embodiment are described with reference to FIG. 1to FIG. 7. FIG. 1 is a block diagram for illustrating a configuration ofthe image pickup apparatus according to the first embodiment.Description is given here of an example of a case in which an imagepickup apparatus 100 according to the first embodiment is an electroniccamera, that is, a digital camera, which is capable of acquiring amoving image and a still image, but the present invention is not limitedthereto.

As illustrated in FIG. 1, the image pickup apparatus 100 includes animage pickup optical system 101, a solid-state image pickup element 102,a drive unit 103, a signal processor 104, a compression/expansion unit105, a controller 106, a light emission unit 107, an operation unit 108,an image display 109, and an image recording unit 110.

The image pickup optical system (lens unit) 101 includes a lens groupfor condensing light reflected from an object to the solid-state imagepickup element 102. The lens group includes a focus lens and a zoomlens. The lens group and microlenses ML (see FIG. 3A to FIG. 4B) to bedescribed later are combined to construct the image pickup opticalsystem. The image pickup optical system 101 includes an opticalmechanism 1011 including an AF mechanism, a zoom drive mechanism, amechanical shutter mechanism, and a diaphragm mechanism. The opticalmechanism 1011 is driven by the drive unit (drive circuit) 103 based ona control signal output from the controller 106.

The solid-state image pickup element 102 includes pixels 201 to bedescribed later and an A/D converter (not shown). The solid-state imagepickup element 102 is, for example, an XY-readout-type CMOS imagesensor. The solid-state image pickup element 102 is driven by the driveunit 103 based on a control signal output from the controller 106. Thesolid-state image pickup element 102 performs image pickup operationssuch as exposure and accumulation, signal readout, and reset, to outputsignals acquired by the image pickup operations, that is, image pickupsignals (hereinafter also referred to as “image signals” and “imagedata”). The solid-state image pickup element 102 includes an AFevaluation value calculator 1021. The AF evaluation value calculator1021 calculates an AF evaluation value, that is, a phase differenceevaluation value and a contrast evaluation value, based on the signalsacquired by the solid-state image pickup element 102, to output thecalculated AF evaluation value to the controller 106.

The signal processor 104 is controlled by the controller 106 to subjectthe image signals acquired by the solid-state image pickup element 102to predetermined signal processing such as white balance adjustmentprocessing, color correction processing, and auto-exposure (AE)processing.

The compression/expansion unit 105 operates under control of thecontroller 106. The compression/expansion unit 105 subjects the imagesignals transmitted from the signal processor 104 to compressionencoding processing, to thereby generate encoded image data. In thecompression encoding processing, a predetermined still image dataformat, for example, a joint photographic coding experts group (JPEG)method, is used. Further, the compression/expansion unit 105 subjectsthe encoded image data transmitted from the controller 106 to expansiondecoding processing, to thereby obtain decoded image data. Thecompression/expansion unit 105 can also subject moving image data to thecompression encoding processing or the expansion decoding processing. Inthe compression encoding processing or the expansion decoding processingwith respect to the moving image data, for example, a moving pictureexperts group (MPEG) method is used.

The controller 106 includes, for example, a central processor (CPU), aread only memory (ROM), and a random access memory (RAM). The CPUexecutes a program stored in the ROM or the like so that the controller106 may generally control the entire image pickup apparatus 100.

When it is determined in the AE processing performed by the signalprocessor 104 that an exposure value of an object is low, the controller106 controls the light emission unit 107 to emit light toward the objectfor illumination of the object. As the light emission unit 107, forexample, a stroboscopic device using a xenon tube or an LED lightemitting device is used.

The operation unit 108 is, for example, various operation keys includinga shutter release button, a lever, a dial, or a touch panel. Theoperation unit 108 transmits an operation signal corresponding to theoperation input performed by a user to the controller 106.

The image display 109 includes a display device (not shown), forexample, a liquid crystal display (LCD), and an interface (not shown)for the display device. The image display 109 displays an imagecorresponding to the image data transmitted from the controller 106 on adisplay screen of the display device.

As the image recording unit 110, a storage medium, for example, aportable semiconductor memory, an optical disc, a hard disk drive (HDD),or a magnetic tape, is used. The encoded image data subjected to thecompression encoding processing by the compression/expansion unit 105 isrecorded in the image recording unit 110 as an image file. The imagerecording unit 110 reads out the image file designated by the controller106 from the storage medium to output the image file to the controller106. The controller 106 causes the compression/expansion unit 105 toperform the expansion decoding processing on the encoded image data readout from the image recording unit 110, to thereby obtain the decodedimage data.

Next, a basic operation of the image pickup apparatus 100 according tothe first embodiment is described. For example, when a still image is tobe taken, the solid-state image pickup element 102 operates as followsbefore the still image is taken. Specifically, the solid-state imagepickup element 102 sequentially subjects the image signals output fromthe pixels 201 to be described later to CDS processing and AGCprocessing, and converts the image signals subjected to the processinginto digital image signals with use of the A/D converter. CDS is anabbreviation for correlated double sampling. Further, AGC is anabbreviation for automatic gain control. The digital image signals thusobtained are transmitted to the AF evaluation value calculator 1021 andthe signal processor 104.

The AF evaluation value calculator 1021 calculates the AF evaluationvalue including the phase difference evaluation value and the contrastevaluation value based on the image signals acquired by the solid-stateimage pickup element 102, to output the calculated AF evaluation valueto the controller 106. The controller 106 determines a feed-back controlamount, that is, a focus lens drive amount based on the AF evaluationvalue, to output the determined feed-back control amount to the driveunit 103. The drive unit 103 drives the focus lens with use of the AFmechanism of the optical mechanism 1011 based on the feed-back controlamount transmitted from the controller 106.

The signal processor 104 subjects the digital image signals output fromthe solid-state image pickup element 102 to, for example, image qualitycorrection processing, to thereby generate, for example, acamera-through image signal, that is, a live view image signal. Thesignal processor 104 transmits the generated live view image signal tothe image display 109 via the controller 106. The image display 109displays a live view image corresponding to the live view image signal.The user can adjust the angle of view (perform framing) while viewingthe live view image displayed on the image display 109.

When the user depresses the shutter release button of the operation unit108 under a state in which the live view image is displayed on the imagedisplay 109, the controller 106 performs the following processing.Specifically, the controller 106 causes the drive unit 103 to controlthe solid-state image pickup element 102 so that image pickup signals ofone frame, that is, digital image signals of one frame, are transmittedfrom the solid-state image pickup element 102 to the signal processor104. The signal processor 104 subjects the digital image signals of oneframe, which are transmitted from the solid-state image pickup element102, to, for example, image quality correction processing, to therebytransmit the digital image signals subjected to the image qualitycorrection processing, that is, image data, to the compression/expansionunit 105. The compression/expansion unit 105 subjects the image data tocompression encoding processing to obtain encoded image data. Theencoded image data obtained by the compression/expansion unit 105 istransmitted to the image recording unit 110 via the controller 106.Thus, the image file of the still image acquired with use of thesolid-state image pickup element 102 is recorded in the image recordingunit 110.

When the image file of the still image recorded in the image recordingunit 110 is to be reproduced, the controller 106 performs the followingprocessing. Specifically, the controller 106 reads out the image filethat is selected by the user via the operation unit 108 from the imagerecording unit 110. Then, the controller 106 transmits the image fileread out from the image recording unit 110 to the compression/expansionunit 105. The compression/expansion unit 105 subjects the image file toexpansion decoding processing to obtain decoded image data. Thecontroller 106 transmits the decoded image data obtained by thecompression/expansion unit 105 to the image display 109. The imagedisplay 109 displays a still image corresponding to the decoded imagedata.

When the moving image data is to be recorded, the controller 106performs the following processing. Specifically, the controller 106causes the drive unit 103 to control the solid-state image pickupelement 102 so that the digital image signals, which are sequentiallyoutput from the solid-state image pickup element 102, may besequentially input to the signal processor 104. The signal processor 104sequentially subjects the sequentially-input digital image signals topredetermined image processing, to thereby generate image data, that is,moving image data. The compression/expansion unit 105 subjects themoving image data to compression encoding processing to obtain encodedmoving image data. The encoded moving image data obtained by thecompression/expansion unit 105 is sequentially transferred to the imagerecording unit 110 via the controller 106, to thereby be recorded in theimage recording unit 110 as a moving image file.

When the moving image file recorded in the image recording unit 110 isto be reproduced, the controller 106 performs the following processing.Specifically, the controller 106 reads out the moving image file that isselected by the user via the operation unit 108 from the image recordingunit 110. The controller 106 transmits the moving image file read outfrom the image recording unit 110 to the compression/expansion unit 105.The compression/expansion unit 105 subjects the moving image file toexpansion decoding processing to obtain decoded moving image data. Thecontroller 106 transmits the decoded moving image data obtained by thecompression/expansion unit 105 to the image display 109. The imagedisplay 109 displays a moving image corresponding to the decoded movingimage data.

FIG. 2A is a perspective view for illustrating a structure of thesolid-state image pickup element included in the image pickup apparatusaccording to the first embodiment. As illustrated in FIG. 2A, thesolid-state image pickup element 102 according to the first embodimentis a stacked image sensor including a first semiconductor chip (a firstsemiconductor substrate) 20 and a second semiconductor chip (a secondsemiconductor substrate) 21, in which the first semiconductor chip 20 isstacked on the second semiconductor chip 21. The first semiconductorchip 20 includes a pixel array 206 in which the plurality of pixels(pixel portions) 201 are arrayed in a two-dimensional matrix manner,that is, a row-column manner. The first semiconductor chip 20 isarranged on a light entering side with respect to the secondsemiconductor chip 21. That is, the first semiconductor chip 20 ispositioned on an optical image reception side with respect to the secondsemiconductor chip 21. The second semiconductor chip 21 has formedthereon a pixel drive circuit (readout circuit, readout unit) includingcolumn scanning circuits 213 a and 213 b and a row scanning circuit 212to be described later. The second semiconductor chip 21 further has theAF evaluation value calculator 1021 formed thereon.

In the first embodiment, the first semiconductor chip 20 on which thepixels 201 are formed and the second semiconductor chip 21 on whichperipheral circuits are formed are different semiconductor chips, andhence a process of forming the pixels 201 and a process of forming theperipheral circuits are separated. Therefore, in the first embodiment,the peripheral circuits or the like may have thinner wiring and higherdensity, and thus the solid-state image pickup element 102 may beincreased in speed, downsized, and sophisticated in function.

FIG. 2B and FIG. 2C are circuit diagrams for illustrating thesolid-state image pickup element according to the first embodiment. FIG.2C is a circuit diagram for illustrating the configuration of the pixel201. As illustrated in FIG. 2B, the plurality of pixels 201 arrayed in atwo-dimensional matrix manner are formed on the first semiconductor chip20. Each of the pixels 201 is connected to a transfer signal line 203, areset signal line 204, and a row selection signal line 205 in ahorizontal direction, and is connected to a column signal line 202 a ora column signal line 202 b in a vertical direction. The pixel 201connected to the column signal line 202 a or 202 b differs depending ona readout row. That is, the pixel 201 located in a row including pixels201SHA and 201SHB (see FIG. 4A) for phase difference detection (forfocus detection) to be described later, that is, a phase difference AFrow (AF row), is connected to the column signal line 202 b. Meanwhile,the pixel 201 located in a row not including the pixels 201SHA and201SHB for phase difference detection, that is, a normal row, isconnected to the column signal line 202 a.

As illustrated in FIG. 2C, the pixel 201 includes a photoelectricconverter (photodiode) PD, a transfer transistor M1, a reset transistorM2, an amplification transistor M3, a selection transistor M4, and afloating diffusion FD. As the transistors M1 to M4, for example,n-channel MOS field-effect transistors (MOSFETs) are used.

The transfer signal line 203 is connected to a gate of the transfertransistor M1. The reset signal line 204 is connected to a gate of thereset transistor M2. The row selection signal line 205 is connected to agate of the selection transistor M4. Those signal lines 203 to 205extend in the horizontal direction, and pixels 201 located in the samerow are driven simultaneously. In this manner, an operation of a linesequential operation rolling shutter or an all-row simultaneousoperation global shutter can be achieved. Further, the column signalline 202 a or the column signal line 202 b is connected to a source ofthe selection transistor M4.

The photoelectric converter PD generates charges through photoelectricconversion. An anode side of the photoelectric converter PD is grounded,and a cathode side of the photoelectric converter PD is connected to asource of the transfer transistor M1. When the transfer transistor M1 isturned on, the charges of the photoelectric converter PD are transferredto the floating diffusion FD. The floating diffusion FD has a parasiticcapacitance, and hence the charges transferred from the photoelectricconverter PD are accumulated in the floating diffusion FD.

A power supply voltage Vdd is applied to a drain of the amplificationtransistor M3, and a gate of the amplification transistor M3 isconnected to the floating diffusion FD. The gate of the amplificationtransistor M3 has a potential corresponding to the charges accumulatedin the floating diffusion FD. The selection transistor M4 is used toselect the pixel 201 from which a signal is read out, and a drain of theselection transistor M4 is connected to a source of the amplificationtransistor M3. Further, the source of the selection transistor M4 isconnected to the column signal line 202 a or the column signal line 202b. When the selection transistor M4 is turned on, an output signalcorresponding to the potential of the gate of the amplificationtransistor M3 is output to the column signal line 202 a or the columnsignal line 202 b. The power supply voltage Vdd is applied to a drain ofthe reset transistor M2, and a source of the reset transistor M2 isconnected to the floating diffusion FD. When the reset transistor M2 isturned on, the potential of the floating diffusion FD is reset to thepower supply voltage Vdd.

The second semiconductor chip 21 includes AD conversion units (columnADC blocks) 211. Each of the AD conversion units 211 is connected to thecolumn signal line 202 a or the column signal line 202 b. The secondsemiconductor chip 21 further includes the row scanning circuit 212 thatscans respective rows, and the column scanning circuit 213 a and 213 bthat scan respective columns. The second semiconductor chip 21 furtherincludes a timing control circuit 214 that controls the operation timingof each of the row scanning circuit 212, the column scanning circuits213 a and 213 b, and the AD conversion units 211 based on a controlsignal output from the controller 106. The second semiconductor chip 21further includes horizontal signal lines 215 a and 215 b fortransferring signals from the AD conversion units 211 in accordance withthe timing controlled by the column scanning circuits 213 a and 213 b.

The second semiconductor chip 21 further includes a switch (signalswitch) 216 that switches an output destination of the digital imagesignals output via the horizontal signal line 215 b. When the switch 216has a first setting, the output destination of the digital image signalsoutput via the horizontal signal line 215 b is set to a phase differenceevaluation value calculator 217 of the AF evaluation value calculator1021. Meanwhile, when the switch 216 has a second setting, the outputdestination of the digital image signals output via the horizontalsignal line 215 b is set to a contrast evaluation value calculator 218of the AF evaluation value calculator 1021 and the signal processor 104.

As described above, the pixel 201 located in the row not including thepixels 201SHA and 201SHB for phase difference detection, that is, thenormal row, is connected to the column signal line 202 a. Therefore, asignal acquired by the pixel 201 located in the normal row is output tothe signal processor 104 via the column signal line 202 a and thehorizontal signal line 215 a. The signal acquired by the pixel 201located in the normal row is used for, for example, an image for liveview.

Meanwhile, as described above, the pixel 201 located in the rowincluding the pixels 201SHA and 201SHB for phase difference detection,that is, the phase difference AF row, is connected to the column signalline 202 b. Therefore, signals SHA or SHB acquired by the pixels 201located in the phase difference AF row are transferred via the columnsignal lines 202 b and the horizontal signal line 215 b. The signals SHAor SHB acquired by the pixels 201SHA or 201SHB for phase differencedetection, which are located in the phase difference AF row, aretransferred to the phase difference evaluation value calculator 217 viathe column signal lines 202 b and the horizontal signal line 215 b.Incidentally, in the phase difference AF row, not only the pixels 201SHAand 201SHB for phase difference detection but also pixels 201G for imagepickup are located (see FIG. 4A). When pixels having specific spectralsensitivity like red (R), green (G), and blue (B) are described,reference symbols of 201R, 201G, and 201B are used, respectively.Further, when the pixels for the phase difference detection aredescribed, reference symbols of 201SHA and 201SHB are used. Further,when general pixels are described, a reference symbol of 201 is used.Signals Gr and Gb acquired by the pixels 201G for image pickup among thepixels 201 located in the phase difference AF row are transmitted to thecontrast evaluation value calculator 218 via the column signal lines 202b and the horizontal signal line 215 b. Further, the signals Gr and Gbacquired by the pixels 201G located in the phase difference AF row aretransmitted to the signal processor 104. For example, at the time oflive view display, the signals Gr and Gb acquired by the pixels 201G forimage pickup, which are located in the phase difference AF row, may beused for interpolation processing of an image signal for live viewdisplay, which is read out while being decimated. A signal transferredto the horizontal signal line 215 a from the pixel 201 located in thenormal row is referred to as “first image signal”. Further, a signaltransferred to the horizontal signal line 215 b from the pixel 201located in the phase difference AF row is referred to as “second imagesignal”.

FIG. 5 is a diagram for illustrating an example of rows selected at thetime of readout in the solid-state image pickup element according to thefirst embodiment. FIG. 5 is an illustration of pixels 201 in 16 rows and6 columns extracted from the plurality of pixels 201 arranged in matrix.Those pixels 201 form a Bayer arrangement. In the first embodiment, inorder that acquisition of the first image signals and acquisition of thesecond image signals can be performed in parallel, rows selected whenthe first image signals are to be acquired and rows selected when thesecond image signals are to be acquired are each set. The first imagesignals are output to the column signal line 202 a. Meanwhile, thesecond image signals are output to the column signal line 202 b. Therows having the row numbers of 1 and 2 and the rows having the rownumbers of 9 and 10 are the rows selected when the second image signalsare to be acquired, which include a pixel group (second pixel group) foracquiring the second image signals. The rows having the row numbers of 3to 8 and the rows having the row numbers of 11 to 16 are the rowsselected when the first image signals are to be acquired, which includea pixel group (first pixel group) for acquiring the first image signals.

The second image signals include signals acquired by the pixels 201SHAand 201SHB for phase difference detection, and hence increasing theframe rate becomes important to achieve a fast autofocus. Therefore, thesecond image signals are set to have a relatively high decimation ratio.Meanwhile, the first image signals are used for live view display andthe like, and hence the image quality becomes important. Therefore, thefirst image signals are set to have a relatively low decimation ratio.Focusing on the rows having the row numbers of 1 to 8, the first imagesignals are obtained by decimating one pixel among the four same-colorpixels arranged in the vertical direction, and the second image signalsare obtained by decimating three pixels among the four same-color pixelsarranged in the vertical direction. When the first image signals are tobe acquired, the first pixel group is read out at a first frame rate.When the second image signals are to be acquired, the second pixel groupis read out at a second frame rate that is higher than the first framerate. Description is given here of an example of a case in which thesecond frame rate is three times as high as the first frame rate.

As described above, in the first embodiment, the rows from which thefirst image signals are read out and the rows from which the secondimage signals are read out are separately set. Therefore, according tothe first embodiment, the first image signal and the second image signalhaving different charge accumulation time periods, different data sizes,and different frame rates can be acquired in parallel. Description hasbeen given of an example of a case in which the second pixel group foracquiring the second image signals is located in the rows having the rownumbers of 1 and 2, and the first pixel group for acquiring the firstimage signals is located in the rows having the row numbers of 3 to 8,but the present invention is not limited thereto. Further, thedecimation ratio for the readout can be set as appropriate.

The analog signal output from the pixel 201 to the column signal line202 a or the column signal line 202 b is converted from analog todigital in the AD conversion unit 211. The column scanning circuit 213 atransmits the digital first image signal output from the AD conversionunit 211 to the signal processor 104 via the horizontal signal line 215a. The first image signal is output to the signal processor 104 via anoutput terminal 222 of the solid-state image pickup element 102.Further, the column scanning circuit 213 b transmits the digital secondimage signal output from the AD conversion unit 211 to the AF evaluationvalue calculator 1021 or the signal processor 104 via the horizontalsignal line 215 b.

The controller 106 uses the signal output from the phase differenceevaluation value calculator 217 of the solid-state image pickup element102, that is, the phase difference evaluation value, to performautofocus control by the phase difference detection method, that is,phase difference focus control (phase difference AF). The controller 106also uses the signal output from the contrast evaluation valuecalculator 218 of the solid-state image pickup element 102, that is, thecontrast evaluation value, to perform autofocus control by the contrastdetection method, that is, contrast focus control (contrast AF).

The phase difference evaluation value calculator 217 performscorrelation calculation with respect to a pair of image signalsgenerated by the signals SHA and SHB output from the plurality of pixels201SHA and 201SHB for phase difference detection, which are located inthe phase difference AF row, to thereby calculate a phase differencerepresenting a relative deviation between the pair of image signals.Then, the phase difference evaluation value calculator 217 calculates adefocus amount Df, which represents an amount of out-of-focus, based onthe phase difference. The controller 106 calculates, based on thedefocus amount Df obtained by the phase difference evaluation valuecalculator 217, the amount to move the focus lens in order to obtain astate close to an in-focus state, that is, a focus lens drive amount toa position near an in-focus position. The controller 106 causes thedrive unit 103 to control the optical mechanism 1011 so that the focuslens is moved by the calculated drive amount.

The contrast evaluation value calculator 218 calculates the contrastevaluation value by extracting, as appropriate, high-frequencycomponents in the signals Gr and Gb output from the pixels 201G forimage pickup, which are located in the phase difference AF row. Thecontroller 106 causes the drive unit 103 to control the opticalmechanism 1011 based on the contrast evaluation value obtained by thecontrast evaluation value calculator 218 so that the focus lens isdriven as appropriate.

FIG. 3A and FIG. 3B are diagrams for illustrating the pixels and theimage pickup optical system. FIG. 3A is a plan view for illustrating thepixels. The four pixels 201 arranged in two rows and two columnsillustrated in FIG. 3A are all pixels for image pickup. At two locationsat the opposing corners of the four pixels 201, pixels 201G havingspectral sensitivity of green (G) are arranged, and pixels 201R and 201Bhaving spectral sensitivity of red (R) and blue (B) are arranged at thetwo remaining locations. Such a pixel arrangement is referred to as“Bayer arrangement”. FIG. 3A is an illustration of one pixel unit beingextracted from the Bayer arrangement. FIG. 3B is a sectional view forillustrating the relationship between the image pickup optical systemand the pixels. FIG. 3B corresponds to a cross section taken along theline A-A of FIG. 3A.

As illustrated in FIG. 3B, the photoelectric converters PD are formed ina semiconductor substrate 301 of the first semiconductor chip 20 of thesolid-state image pickup element 102. The photoelectric converters PDare formed so as to correspond to the respective plurality of pixels 201formed in matrix. A multi-layer wiring structure 302 is formed on thesemiconductor substrate 301 having the photoelectric converters PDformed therein. The multi-layer wiring structure 302 includes a wiringlayer CL and an insulating layer 303. The wiring layer CL forms signallines for transmitting various signals in the solid-state image pickupelement 102. A color filter layer 304 is formed on the multi-layerwiring structure 302. The color filter layer 304 includes a red (R)color filter CF_(R), a green (G) color filter CF_(C), and a blue (B)color filter CF_(B). Those color filters CF_(R), CF_(G), and CF_(B) areformed so as to correspond to the respective pixels 201. In FIG. 3A, Rconceptually represents a location where the R color filter CF_(R) isarranged, G conceptually represents a location where the G color filterCF_(G) is arranged, and B conceptually represents a location where the Bcolor filter CF_(B) is arranged.

On the color filter layer 304, the microlenses ML, that is,on-semiconductor chip microlenses, are arranged. The microlenses ML areformed so as to correspond to the respective pixels 201. The microlensesML and the photoelectric converters PD are configured such that lightfluxes 305 passing through an exit pupil EP of an image pickup opticalsystem TL can be introduced as effectively as possible. In other words,the microlenses ML are formed so that a conjugate relationship issatisfied between the exit pupil EP of the image pickup optical systemTL and the photoelectric converters PD. Further, an effective area ofthe photoelectric converter PD is designed to be large. In FIG. 3B, thelight fluxes 305 entering the R pixel 201 are illustrated as an example,but there are similarly light fluxes entering the G pixel 201 and lightfluxes entering the B pixel 201. As described above, the exit pupil EPcorresponding to the RGB pixels 201 for image pickup has a largediameter, and thus the light fluxes from the object reach the pixels 201efficiently. Therefore, an image signal having a high S/N ratio can beobtained in each of the pixels 201.

FIG. 4A and FIG. 4B are diagrams for illustrating the pixels and theimage pickup optical system. FIG. 4A is a plan view of the pixels. Whenan image signal is to be obtained, the output from the pixel 201G havingthe spectral sensitivity of G is a main component of brightnessinformation. Human image recognition characteristics are sensitive tobrightness information, and hence when the output from the pixel 201Ghaving the spectral sensitivity of G is defective, the image qualitydeterioration is easily recognized. Meanwhile, the R pixel 201R (seeFIG. 3A and FIG. 3B) and the B pixel 201B (see FIG. 3A and FIG. 3B) arepixels mainly for acquiring color information, and humans are relativelyinsensitive to color information. Therefore, even when the R and Bpixels 201R and 201B for acquiring the color information are slightlydefective, humans are less liable to notice the image qualitydeterioration. In view of this, in the first embodiment, two G pixels201G among the four pixels 201 arranged in two rows and two columns areleft as the pixels for image pickup, and the pixels 201SHA and 201SHBfor phase difference detection are arranged at positions of the R pixel201R and the B pixel 201B, respectively.

In FIG. 4A, SHA conceptually represents the pixel 201SHA for phasedifference detection, which is arranged at the position of the R pixel201R, and SHB conceptually represents the pixel 201SHB for phasedifference detection, which is arranged at the position of the B pixel201B. As described above, in the phase difference AF row, the pixels201SHA and 201SHB for phase difference detection are dispersedlyarranged. Further, as described above, in the phase difference AF row,not only the pixels 201SHA and 201SHB for phase difference detection butalso the pixels 201G for image pickup are located. The plurality ofpixels 201SHA for phase difference detection construct a first pixelgroup for phase difference detection. The plurality of pixels 201SHB forphase difference detection construct a second pixel group for phasedifference detection.

FIG. 4B corresponds to a cross section taken along the line B-B of FIG.4A. Also in the pixels 201SHA and 201SHB for phase difference detection,similarly to the pixels 201R, 201G, and 201B for image pickup, thephotoelectric converters PD are formed in the semiconductor substrate301. The signals from the pixels 201SHA and 201SHB for phase differencedetection are not used for the image, and hence transparent films (whitefilms) CF_(W) are arranged for the pixels 201SHA and 201SHB for phasedifference detection instead of the color filters CF_(R) and CF_(B) forcolor separation. In the pixels 201SHA and 201SHB for phase differencedetection, the wiring layer CL constructs a light shielding portionhaving opening portions OP_(HA) and OP_(HB) in order to achieve pupilseparation.

In order to obtain pupil separation in an x direction, the openingportions OP_(HA) and OP_(HB) are each deviated in the x direction withrespect to the center of the microlens ML. The opening portion OP_(HA)of the pixel 201SHA for phase difference detection is deviated in a −xdirection with respect to the center of the microlens ML. Therefore, thephotoelectric converter PD of the pixel 201SHA for phase differencedetection receives the light flux that has passed through a +x-sidepupil region, that is, a first pupil region EP_(HA), of a plurality ofpupil regions EP_(HA) and EP_(HB) included in the exit pupil of theimage pickup optical system TL.

The pixel 201SHA acquires a signal corresponding to the light fluxpassing through the first pupil region EP_(HA) of the exit pupil of theimage pickup optical system TL. Meanwhile, the opening portion OP_(HB)of the pixel 201SHB for phase difference detection is deviated in the +xdirection with respect to the center of the microlens ML. Therefore, thephotoelectric converter PD of the pixel 201SHB for phase differencedetection receives the light flux that has passed through a −x-sidepupil region, that is, a second pupil region EP_(HB), of the pluralityof pupil regions EP_(HA) and EP_(HB) included in the exit pupil of theimage pickup optical system TL. The pixel 201SHB acquires a signalcorresponding to the light flux passing through the second pupil regionEP_(HB) of the exit pupil of the image pickup optical system TL.

An object image acquired by the plurality of pixels 201SHA for phasedifference detection regularly arranged in the x direction, that is, thefirst pixel group for phase difference detection, is referred to as“first image”. Further, an object image acquired by the plurality ofpixels 201SHB for phase difference detection regularly arranged in the xdirection, that is, the second pixel group for phase differencedetection, is referred to as “second image”. Further, a relativedeviation amount between the first image and the second image, that is,the phase difference, is detected so that the defocus amount Df of thefocus lens with respect to the object can be calculated based on thephase difference.

Signals output from the pixels 201 to the column signal lines 202 a and202 b are converted from analog to digital by the AD conversion units211. The signals converted into digital by the AD conversion units 211are output from the AD conversion units 211 to the horizontal signallines 215 a and 215 b by the column scanning circuits 213 a and 213 b,respectively. The signals output to the horizontal signal line 215 a,that is, the signals from the pixels 201 located in the normal row, areoutput to the signal processor 104 located outside of the solid-stateimage pickup element 102. Meanwhile, the signals output to thehorizontal signal line 215 b are output via the switch 216. When theswitch 216 has the first setting, the signals SHA and SHB from thepixels 201SHA and 201SHB for phase difference detection are output tothe phase difference evaluation value calculator 217. When the switch216 has the second setting, the signals from the pixels 201 for imagepickup, which are located in the phase difference AF row, are output tothe contrast evaluation value calculator 218 and the signal processor104.

The phase difference evaluation value calculator 217 calculates thephase difference evaluation value through correlation calculation basedon the signals SHA and SHB transmitted to the phase differenceevaluation value calculator 217 via the horizontal signal line 215 b andthe switch 216. That is, the phase difference evaluation valuecalculator 217 calculates the phase difference evaluation value forfocus detection by the phase difference detection method based on thesignals SHA and SHB from the pixels 201SHA and 201SHB. The phasedifference evaluation value calculator 217 outputs the phase differenceevaluation value to the controller 106 as soon as the calculation of thephase difference evaluation value is completed regardless of whether ornot the output of the image signal acquired by the pixel array 206 tothe signal processor 104 is completed. The phase difference evaluationvalue is output to the controller 106 via an output terminal 219 of thesolid-state image pickup element 102.

The contrast evaluation value calculator 218 calculates the contrastevaluation value through contrast calculation based on the signalstransmitted to the contrast evaluation value calculator 218 via thehorizontal signal line 215 b and the switch 216, more specifically, forexample, the signals Gr and Gb. That is, the contrast evaluation valuecalculator 218 calculates the contrast evaluation value for focusdetection by the contrast detection method based on the signals Gr andGb from the pixels 201G for image pickup. The contrast evaluation valuecalculator 218 outputs the contrast evaluation value to the controller106 as soon as the calculation of the contrast evaluation value iscompleted regardless of whether or not the output of the image signalacquired by the pixel array 206 to the signal processor 104 iscompleted. The contrast evaluation value is output to the controller 106via an output terminal 220 of the solid-state image pickup element 102.

As described above, when the switch 216 has the second setting, thesignals Gr and Gb from the pixels 201G for image pickup, which arelocated in the phase difference AF row, are output to the signalprocessor 104 via the horizontal signal line 215 b and the switch 216.The signals Gr and Gb are output to the signal processor 104 via anoutput terminal 221 of the solid-state image pickup element 102. Thesignals Gr and Gb transmitted to the signal processor 104 via thehorizontal signal line 215 b and the switch 216 may be used for, forexample, interpolation processing of the live view image that is readout while being decimated.

In this case, a path through which the first image signal is output viathe column signal line 202 a and the horizontal signal line 215 a isreferred to as “first channel CH1”, and a path through which the secondimage signal is output via the column signal line 202 b and thehorizontal signal line 215 b is referred to as “second channel CH2”.

FIG. 6 is a time chart for illustrating an operation of the image pickupapparatus according to the first embodiment. FIG. 6 is an illustrationof an operation in a mode in which live view display is performed whilean autofocus operation is performed, that is, an AF evaluation mode.

The AF evaluation mode is started when an AF control signal from thecontroller 106 is turned on. A timing at which the AF control signal isturned on is set to T0. At the timing T0, a vertical synchronizationsignal VD falls, that is, the level changes from a High level to a Lowlevel. In synchronization with the vertical synchronization signal VD,acquisition of the first image signals, that is, the signals from thepixels 201 located in the normal row, and acquisition of the secondimage signals, that is, the signals from the pixels 201 located in thephase difference AF row, are started. As described above, the firstimage signals are output via the first channel CH1, and the second imagesignals are output via the second channel CH2.

During a period between T0 and TF1, the signals SHA and SHB are read outfrom the pixels 201SHA and 201SHB for phase difference detection, whichare located in the phase difference AF row. The readout of the signalsfrom the pixels 201SHA and 201SHB for phase difference detection, whichare located in the phase difference AF row, is performed in parallel tothe readout of the signals from the pixels 201 located in the normalrow, that is, the readout of signals for live view. The signals SHA andSHB for phase difference evaluation, which are read out from the pixels201SHA and 201SHB, are transmitted to the phase difference evaluationvalue calculator 217 via the horizontal signal line 215 b and the switch216. Then, during a period between TF1 and TF2, the phase differenceevaluation value calculator 217 calculates the phase differenceevaluation value.

During a period between TF2 and TF3, the phase difference evaluationvalue calculated by the phase difference evaluation value calculator 217is output to the controller 106. Further, during the period between TOand TF1, the signals Gr and Gb are read out from the pixels 201G forimage pickup, which are located in the phase difference AF row. Thereadout of the signals Gr and Gb from the pixels 201G for image pickup,which are located in the phase difference AF row, is performed inparallel to the readout of the signals from the pixels 201 located inthe normal row, that is, the readout of the signals for live view. Thesignals Gr and Gb for contrast evaluation, which are read out from thepixels 201G, are transmitted to the contrast evaluation value calculator218 via the horizontal signal line 215 b and the switch 216. Then,during a period between TF1 and TF2C, the contrast evaluation valuecalculator 218 calculates the contrast evaluation value. During a periodbetween TF2C and TF3C, the contrast evaluation value calculated by thecontrast evaluation value calculator 218 is output to the controller106.

As illustrated in FIG. 6, the vertical synchronization signal VD falls,that is, changes to the Low level, at each predetermined time intervalΔT. The vertical synchronization signal that has changed to the Lowlevel is returned to the High level after an elapse of a predeterminedtime period. The predetermined time interval ΔT corresponds to a periodrequired for acquiring an image for live view of one frame, that is, aone-frame live view signal acquisition period. In the first embodiment,during the one-frame live view signal acquisition period, AF evaluationis performed three times. Acquisition of the signal for AF evaluation,that is, acquisition of the second image signals, is performedindependently and in parallel to acquisition of the signals for liveview, that is, acquisition of the first image signals. The controller106 determines whether or not a desired in-focus state can be obtained.When the desired in-focus state can be obtained, the AF control signalis turned off, that is, returned to the Low level. In FIG. 6, a timingT1 indicates the timing at which the AF control signal is turned off.When the AF control signal is turned off, the AF evaluation is canceled,but the image for live view is continuously acquired.

FIG. 7 is a flow chart for illustrating the operation of the imagepickup apparatus according to the first embodiment.

When the user operates the operation unit 108 to turn on the power ofthe image pickup apparatus 100, the controller 106 sets the image pickupapparatus 100 to a standby state. The controller 106 determines whetheror not the image pickup apparatus 100 is required to be operated in theAF evaluation mode based on the operation performed by the user on theoperation unit 108 or the like (Step S701). When the image pickupapparatus 100 is not required to be operated in the AF evaluation mode(NO in Step S701), acquisition of the signals for live view is started(Step S702). Then, a live view image obtained by the acquisition of thesignals for live view is displayed on the image display 109 (Step S717).On the other hand, when the image pickup apparatus 100 is required to beoperated in the AF evaluation mode (YES in Step S701), the controller106 turns on the AF control signal (Step S703), and sets the number oftimes n of acquiring the signals for phase difference evaluation to 0(Step S704).

After that, the controller 106 causes the solid-state image pickupelement 102 to start the acquisition of the signals for live view (StepS718), and causes the solid-state image pickup element 102 to start theacquisition of the signals for phase difference evaluation (Step S705).After the controller 106 causes the solid-state image pickup element 102to start the acquisition of the signals for phase difference evaluation,the controller 106 increments the number of times n of acquiring thesignals for phase difference evaluation (Step S706). After that, thephase difference evaluation value is calculated by the phase differenceevaluation value calculator 217 (Step S707), and the contrast evaluationvalue is calculated by the contrast evaluation value calculator 218(Step S708).

In the calculation of the phase difference evaluation value (Step S707),the phase difference evaluation value calculator 217 calculates thephase difference evaluation value based on the signals SHA and SHB forphase difference evaluation acquired in Step S705, to thereby obtain thedefocus amount Df. The phase difference evaluation value calculator 217outputs the phase difference evaluation value, specifically, the defocusamount Df, to the controller 106 as soon as the calculation of thedefocus amount Df is completed. In the calculation of the contrastevaluation value (Step S708), the contrast evaluation value calculator218 calculates a contrast evaluation value AF_K based on the signals Grand Gb for contrast evaluation acquired in Step S705. The contrastevaluation value calculator 218 outputs the contrast evaluation valueAF_K to the controller 106 as soon as the calculation of the contrastevaluation value AF_K is completed.

In Step S709, the controller 106 determines whether or not the defocusamount Df calculated in Step S707 falls within a range of a desireddefocus amount based on Expression (1).

Df_min<Df<Df_max  (1)

In Expression (1), Df_min and Df_max represent the minimum value and themaximum value of the desired defocus amount Df, respectively, and are,for example, stored in advance in the ROM of the controller 106. Thosevalues Df_min and Df_max may be determined at the time of design oradjustment.

When the defocus amount Df does not satisfy Expression (1) (NO in StepS709), the processing proceeds to Step S710. In Step S710, thecontroller 106 determines the feed-back control amount, that is, thefocus lens drive amount, based on the defocus amount Df. Then, thecontroller 106 causes the drive unit 103 to control the opticalmechanism 1011, to thereby drive the focus lens in the image pickupoptical system 101 (phase difference AF control). After that, theprocessing proceeds to Step S713. In Step S713, the controller 106determines whether or not the number of times n of acquiring the signalsfor phase difference evaluation is 3. When the number of times n ofacquiring the signals for phase difference evaluation is not 3 (NO inStep S713), the processing returns to Step S705. On the other hand, whenthe number of times n of acquiring the signals for phase differenceevaluation is 3 (YES in Step S713), live view display is performed (StepS714), and the processing returns to Step S704.

When the defocus amount Df satisfies Expression (1) (YES in Step S709),the controller 106 determines whether or not the contrast evaluationvalue AF_K obtained in Step S708 falls within a range of a desiredcontrast amount based on Expression (2).

K_min<AF_K<K_max  (2)

In Expression (2), K_min and K_max are the minimum value and the maximumvalue of the desired contrast evaluation value AF_K, respectively, andare, for example, stored in the ROM of the controller 106. Those valuesK_min and K_max may be determined at the time of design or adjustment.

When the contrast evaluation value AF_K does not satisfy Expression (2)(NO in Step S711), the processing proceeds to Step S712. In Step S712,the controller 106 determines the feed-back control amount, that is, thefocus lens drive amount, based on the contrast evaluation value AF_K.Then, the controller 106 causes the drive unit 103 to control theoptical mechanism 1011, to thereby drive the focus lens in the imagepickup optical system 101 (contrast AF control). After that, theprocessing proceeds to Step S713. When the contrast evaluation valueAF_K satisfies Expression (2) (YES in Step S711), the controller 106turns off the AF control signal (Step S715). After that, the controller106 ends the acquisition of the signals for phase difference evaluation(Step S716). The controller 106 displays the signals for live viewacquired in Step S718 on the image display 109 (Step S717), and thencauses the image pickup apparatus 100 to shift to the standby state.

As described above, according to the first embodiment, when the phasedifference evaluation value is calculated by the phase differenceevaluation value calculator 217, the solid-state image pickup element102 outputs the phase difference evaluation value regardless of whetheror not the output of the signals for live view is completed. Further,according to the first embodiment, when the contrast evaluation value iscalculated by the contrast evaluation value calculator 218, thesolid-state image pickup element 102 outputs the contrast evaluationvalue regardless of whether or not the output of the signals for liveview is completed. According to the first embodiment, the phasedifference evaluation value and the contrast evaluation value are outputto the controller 106 regardless of whether or not the output of thesignals for live view is completed. Thus, an image pickup apparatuscapable of achieving a fast autofocus with high accuracy can beprovided.

Second Embodiment

A solid-state image pickup element, a method of controlling asolid-state image pickup element, and an image pickup apparatusaccording to a second embodiment are described with reference to FIG. 8and FIG. 9. Like components in the solid-state image pickup element, themethod of controlling a solid-state image pickup element, and the imagepickup apparatus according to the first embodiment illustrated in FIG. 1to FIG. 7 are denoted by like reference symbols to omit or simplify thedescription.

The image pickup apparatus 100 according to the second embodimentcontinuously acquires still images, that is, perform continuous shootingof still images.

FIG. 8 is a time chart for illustrating an operation of the image pickupapparatus according to the second embodiment. A still image continuousshooting mode, which is a mode of continuously acquiring the stillimages, is started when a still image continuous shooting control signalfrom the controller 106 is turned on. The controller 106 causes thedrive unit 103 to control the mechanical shutter mechanism and thediaphragm mechanism of the optical mechanism 1011, and causes the driveunit 103 to control the solid-state image pickup element 102.Specifically, a reset operation is performed on the pixels 201, and amechanical shutter (shutter) is opened so that exposure to the pixels201 is started. After the exposure to the pixels 201 is started,photoelectric conversion by the photoelectric converters PD is started.

After an elapse of such a predetermined exposure time period thatsatisfies an exposure condition set in advance, the controller 106causes the drive unit 103 to close the shutter. When the shutter isclosed, the exposure to the pixels 201 is completed. After the exposureto the pixels 201 is completed, acquisition of the first image signals,that is, the signals for the still image, and acquisition of the secondimage signals, that is, the signals for phase difference evaluation, arestarted. The timing at which the acquisition of the first image signalsand the acquisition of the second image signals are started is TF0. Thefirst image signals are output via the first channel CH1, and the secondimage signals are output via the second channel CH2. The rows selectedwhen the first image signals are to be acquired and the rows selectedwhen the second image signals are to be acquired may be the same asthose in the case of the first embodiment described above with referenceto FIG. 5, for example.

During a period between TF0 and TF1, the signals SHA and SHB for phasedifference evaluation are transmitted to the phase difference evaluationvalue calculator 217 via the horizontal signal line 215 b and the switch216. Then, during a period between TF1 and TF2, the phase differenceevaluation value calculator 217 calculates the phase differenceevaluation value. During a period between TF2 and TF3, the phasedifference evaluation value calculated by the phase differenceevaluation value calculator 217 is output to the controller 106.Further, during the period between TF0 and TF1, the signals for contrastevaluation are transmitted to the contrast evaluation value calculator218 via the horizontal signal line 215 b and the switch 216. Then,during a period between TF1 and TF2C, the contrast evaluation valuecalculator 218 calculates the contrast evaluation value.

During a period between TF2C and TF3C, the contrast evaluation valuecalculated by the contrast evaluation value calculator 218 is output tothe controller 106. The controller 106 causes the drive unit 103 todrive the focus lens of the image pickup optical system 101 based on thephase difference evaluation value and the contrast evaluation value.When the drive of the focus lens is completed, and the still imagecontinuous shooting mode is not ended, the processing proceeds to theacquisition of the next still image.

FIG. 9 is a flow chart for illustrating the operation of the imagepickup apparatus according to the second embodiment. First, thecontroller 106 causes the image pickup apparatus 100 to shift from thestandby state to the still image continuous shooting mode based on theoperation input performed by the user on the operation unit 108. In StepS901, the controller 106 causes the drive unit 103 to open themechanical shutter of the optical mechanism 1011. In this manner, theexposure to the pixels 201 is started, and the photoelectric conversionby the photoelectric converters PD is started so that charges arestarted to be accumulated (Step S902). After an elapse of apredetermined exposure time period, the controller 106 causes the driveunit 103 to close the mechanical shutter of the optical mechanism 1011(Step S903).

Next, the controller 106 causes the solid-state image pickup element 102to start the acquisition of the signals for the still image (Step S905),and causes the solid-state image pickup element 102 to start theacquisition of the signals for phase difference evaluation (Step S904).After that, the phase difference evaluation value is calculated by thephase difference evaluation value calculator 217 (Step S906), and thecontrast evaluation value is calculated by the contrast evaluation valuecalculator 218 (Step S907). In the calculation of the phase differenceevaluation value (Step S906), the phase difference evaluation valuecalculator 217 calculates the phase difference evaluation value based onthe signals SHA and SHB for phase difference evaluation acquired in StepS904, to thereby obtain the defocus amount Df.

The phase difference evaluation value calculator 217 outputs the phasedifference evaluation value, specifically, the defocus amount Df, to thecontroller 106 as soon as the calculation of the phase differenceevaluation value is completed. In the calculation of the contrastevaluation value (Step S907), the contrast evaluation value calculator218 calculates the contrast evaluation value AF_K based on the signalsGr and Gb for image pickup acquired in Step S904. The contrastevaluation value calculator 218 outputs the contrast evaluation valueAF_K to the controller 106 as soon as the calculation of the contrastevaluation value AF_K is completed.

In Step S908, the controller 106 determines whether or not the defocusamount Df calculated in Step S906 falls within a range of a desireddefocus amount based on Expression (1) described above in the firstembodiment. When the defocus amount Df does not satisfy Expression (1)(NO in Step S908), the processing proceeds to Step S909. In Step S909,the controller 106 determines the feed-back control amount, that is, thefocus lens drive amount, based on the defocus amount Df. Then, thecontroller 106 causes the drive unit 103 to control the opticalmechanism 1011, to thereby drive the focus lens in the image pickupoptical system 101 (phase difference AF control). After that, theprocessing proceeds to Step S912. When the defocus amount Df satisfiesExpression (1) (YES in Step S908), the processing proceeds to Step S910.

In Step S910, the controller 106 determines whether or not the contrastevaluation value AF_K obtained in Step S907 falls within a range of adesired contrast amount based on Expression (2) described above in thefirst embodiment. When the contrast evaluation value AF_K does notsatisfy Expression (2) (NO in Step S910), the processing proceeds toStep S911. In Step S911, the controller 106 determines the feed-backcontrol amount, that is, the focus lens drive amount, based on thecontrast evaluation value AF_K. The controller 106 causes the drive unit103 to control the optical mechanism 1011, to thereby drive the focuslens in the image pickup optical system 101 (contrast AF control). Afterthat, the processing proceeds to Step S912. When the contrast evaluationvalue AF_K satisfies Expression (2) (YES in Step S910), the processingproceeds to Step S912.

In Step S912, the controller 106 determines whether or not to end thestill image continuous shooting mode in accordance with the operationinput performed by the user on the operation unit 108. When the stillimage continuous shooting mode is not to be ended (NO in Step S912), theprocessing returns to Step S901 to start the acquisition of the nextstill image. On the other hand, when the still image continuous shootingmode is to be ended (YES in Step S912), the controller 106 causes theimage pickup apparatus 100 to shift to the standby state withoutacquiring the next still image.

As described above, according to the second embodiment, when the phasedifference evaluation value is calculated by the phase differenceevaluation value calculator 217, the solid-state image pickup element102 outputs the phase difference evaluation value regardless of whetheror not the output of the signals for the still image is completed.Further, according to the second embodiment, when the contrastevaluation value is calculated by the contrast evaluation valuecalculator 218, the solid-state image pickup element 102 outputs thecontrast evaluation value regardless of whether or not the output of thesignals for the still image is completed. According to the secondembodiment, the phase difference evaluation value and the contrastevaluation value are output to the controller 106 regardless of whetheror not the output of the signals for the still image is completed. Thus,an image pickup apparatus capable of achieving a fast autofocus withhigh accuracy can be provided.

Third Embodiment

A solid-state image pickup element, a method of controlling asolid-state image pickup element, and an image pickup apparatusaccording to a third embodiment are described with reference to FIG. 10to FIG. 11D. Like components in the solid-state image pickup element,the method of controlling a solid-state image pickup element, and theimage pickup apparatus according to the first and second embodimentsillustrated in FIG. 1 to FIG. 9 are denoted by like reference symbols toomit or simplify the description.

The solid-state image pickup element according to the third embodimentoutputs the image signal and the AF evaluation value via a common outputterminal.

FIG. 10 is a circuit diagram for illustrating the solid-state imagepickup element according to the third embodiment. As illustrated in FIG.10, in the third embodiment, an output unit 1001 is provided. Thesignals transferred to the horizontal signal line 215 a from the pixels201 located in the normal row are input to the output unit 1001.Further, the phase difference evaluation value output from the phasedifference evaluation value calculator 217 of the AF evaluation valuecalculator 1021 is input to the output unit 1001. Further, the contrastevaluation value output from the contrast evaluation value calculator218 of the AF evaluation value calculator 1021 is input to the outputunit 1001. Further, the signals Gr and Gb transferred to the horizontalsignal line 215 b from the pixels 201G for image pickup (see FIG. 4A andFIG. 4B) located in the phase difference AF row are input to the outputunit 1001. The output unit 1001 outputs the image signal and the AFevaluation value to the controller 106 via a common output terminal1002. The image signal may be an image signal for live view as describedabove in the first embodiment, or may be an image signal for the stillimage as described above in the second embodiment.

FIG. 11A is a diagram for conceptually illustrating a layout of a pixelarray in the image pickup element according to the third embodiment.FIG. 11A is an illustration of a part extracted from the pixel array206. The pixel array 206 has a region 1101 in which a plurality ofnormal rows, which are rows not including the pixels 201SHA and 201SHBfor phase difference detection, are arranged. Further, the pixel array206 includes phase difference AF rows 1102 a and 1102 b including thepixels 201SHA and 201SHB for phase difference detection, respectively.The pixels 201SHA and 201SHB may correspond to respective dividedregions (not shown) of the pixel array 206 divided into a plurality ofsections, or may correspond to AF frames. For example, the readout ofthe signal from the phase difference AF row 1102 a is performed, thenthe readout of the signal from the phase difference AF row 1102 b isperformed, and then the readout of the signals from the plurality ofnormal rows included in the region 1101 is sequentially performed.

FIG. 11B is a diagram for conceptually illustrating signals to be outputfrom the image pickup element according to the third embodiment. Asillustrated in FIG. 11B, a signal 1105 a read out from the phasedifference AF row 1102 a is output to the controller 106 via the outputunit 1001. After that, a signal 1105 b read out from the phasedifference AF row 1102 b is output to the controller 106 via the outputunit 1001. After that, the output to the controller 106 of a signal 1104read out from the plurality of normal rows located in the region 1101 isstarted.

Before the readout of the signals from the plurality of normal rowslocated in the region 1101 is completed, as soon as the phase differenceevaluation value is calculated by the phase difference evaluation valuecalculator 217, the phase difference evaluation value is input to theoutput unit 1001. Further, before the readout of the signals from theplurality of normal rows located in the region 1101 is completed, assoon as the contrast evaluation value is calculated by the contrastevaluation value calculator 218, the contrast evaluation value is inputto the output unit 1001. At a stage before the output of the signalsread out from the plurality of normal rows located in the region 1101 iscompleted, that is, at a stage before the output of the image signal1104 is completed, the output unit 1001 outputs the phase differenceevaluation value and the contrast evaluation value to the controller 106as soon as those values are acquired.

The AF evaluation value, that is, the phase difference evaluation valueand the contrast evaluation value, are calculated for each of theregions divided so as to correspond to the phase difference AF rowsarranged evenly in the pixel array 206, and the number of the regions isroughly from 10 to 30, for example. Further, the data length of the AFevaluation value is about 8 bits, for example. The data amount of the AFevaluation value is extremely smaller than those of the signals read outfrom the normal row, that is, the signal for live view and the signalfor the still image, and hence the data of the AF evaluation value canbe inserted as invalid data that does not affect the signal for liveview and the signal for the still image. The timing to insert and outputthe data of the AF evaluation value can be set to a timing within aperiod in which the signal read out from the normal row is not output tothe controller 106, for example, a timing corresponding to a horizontalblanking period. FIG. 11B is a diagram for conceptually illustrating astate in which a phase difference evaluation value 1106 and a contrastevaluation value 1107 are inserted in the horizontal blanking period.

FIG. 11C is a time chart for illustrating an example of an operation ofthe image pickup apparatus according to the third embodiment. FIG. 11Cis an illustration of a case in which an operation is performed in themode of performing live view display while the autofocus operation isperformed, that is, the AF evaluation mode. As described above in thefirst embodiment, first, the acquisition of the signals for live viewand the acquisition of the signals for phase difference evaluation areperformed in parallel. When the phase difference evaluation valueobtained by the phase difference evaluation value calculator 217 isinput to the output unit 1001, the output unit 1001 inserts and outputsthe phase difference evaluation value to a part of the horizontalblanking period of the signals for live view as invalid data. When thecontrast evaluation value obtained by the contrast evaluation valuecalculator 218 is input to the output unit 1001, the output unit 1001inserts and outputs the contrast evaluation value to a part of thehorizontal blanking period of the signals for live view as invalid data.

FIG. 11D is a time chart for illustrating another example of theoperation of the image pickup apparatus according to the thirdembodiment. FIG. 11D is an illustration of a case in which an operationis performed in the still image continuous shooting mode, which is amode of continuously acquiring the still images. As described above inthe second embodiment, the acquisition of the signals for the stillimage and the acquisition of the signals for phase difference evaluationare performed in parallel. When the phase difference evaluation valueobtained by the phase difference evaluation value calculator 217 isinput to the output unit 1001, the output unit 1001 inserts and outputsthe phase difference evaluation value to a part of the horizontalblanking period of the signals for the still image as invalid data.Further, when the contrast evaluation value obtained by the contrastevaluation value calculator 218 is input to the output unit 1001, theoutput unit 1001 inserts and outputs the contrast evaluation value to apart of the horizontal blanking period of the signals for the stillimage as invalid data.

The controller 106 drives the focus lens based on the phase differenceevaluation value output from the output unit 1001. Further, thecontroller 106 drives the focus lens based on the contrast evaluationvalue output from the output unit 1001. When the drive of the focus lensis completed, and the still image continuous shooting mode is not ended,the processing proceeds to the acquisition of the next still image.

FIG. 12A is a diagram for conceptually illustrating signals to be outputfrom a solid-state image pickup element according to ReferenceExample 1. As illustrated in FIG. 12A, in the image pickup apparatusaccording to Reference Example 1, the phase difference evaluation value1106 and the contrast evaluation value 1107 are added after the imagesignal (image data) 1104. That is, in the image pickup apparatusaccording to the Reference Example 1, the phase difference evaluationvalue 1106 and the contrast evaluation value 1107 are arranged at thetail of the image signal 1104.

FIG. 12B is a time chart for illustrating an example of an operation ofan image pickup apparatus according to Reference Example 1. FIG. 12B isan illustration of a case in which an operation is performed in the modeof performing live view display while the autofocus operation isperformed, that is, the AF evaluation mode. Also in Reference Example 1,the acquisition of the signals for phase difference evaluation and theacquisition of the signals for contrast evaluation are performed inparallel to the acquisition of the signals for live view. Also in theimage pickup apparatus according to Reference Example 1, at a stagebefore the output of the signals for live view is completed, thecalculation of the phase difference evaluation value and the contrastevaluation value is completed. In the image pickup apparatus accordingto Reference Example 1, after the output of the signals for live view iscompleted, the phase difference evaluation value and the contrastevaluation value are output. Therefore, in Reference Example 1, even ifthe phase difference evaluation value and the contrast evaluation valueare calculated at a stage before the output of the signals for live viewis completed, a fast autofocus cannot be achieved.

FIG. 12C is a time chart for illustrating another example of theoperation of the image pickup apparatus according to ReferenceExample 1. FIG. 12C is an illustration of a case in which an operationis performed in the still image continuous shooting mode, which is amode of continuously acquiring the still images. Also in ReferenceExample 1, the acquisition of the signals for phase differenceevaluation and the acquisition of the signals for contrast evaluationare performed in parallel to the acquisition of the signals for thestill image. Also in the image pickup apparatus according to ReferenceExample 1, at a stage before the output of the signals for the stillimage is completed, the calculation of the phase difference evaluationvalue and the contrast evaluation value is completed. In the imagepickup apparatus according to Reference Example 1, after the output ofthe signals for the still image is completed, the phase differenceevaluation value and the contrast evaluation value are output.Therefore, in Reference Example 1, even if the phase differenceevaluation value and the contrast evaluation value are calculated at astage before the output of the signals for the still image is completed,a fast autofocus cannot be achieved.

In contrast, in the third embodiment, when the phase differenceevaluation value is calculated, similarly to the image pickup apparatusaccording to the first and second embodiments, the solid-state imagepickup element 102 outputs the phase difference evaluation valueregardless of whether or not the output of the image signal iscompleted. Further, in the third embodiment, when the contrastevaluation value is calculated, similarly to the image pickup apparatusaccording to the first and second embodiments, the solid-state imagepickup element 102 outputs the contrast evaluation value regardless ofwhether or not the output of the image signal is completed. Therefore,according to the third embodiment, similarly to the image pickupapparatus according to the first and second embodiments, it is possibleto provide the image pickup apparatus capable of achieving a fastautofocus with high accuracy. In addition, according to the thirdembodiment, the image signal and the AF evaluation value are output viathe common output terminal 1002, and hence the third embodiment cancontribute to reduction in cost or the like.

Fourth Embodiment

A solid-state image pickup element, a method of controlling asolid-state image pickup element, and an image pickup apparatusaccording to a fourth embodiment are described with reference to FIG. 13to FIG. 17. Like components in the solid-state image pickup element andthe like according to the first to third embodiments illustrated in FIG.1 to FIG. 11D are denoted by like reference symbols to omit or simplifythe description.

The configuration of the image pickup apparatus according to the fourthembodiment is similar to the configuration of the image pickup apparatusaccording to the first embodiment described above with reference toFIG. 1. Further, the structure of the solid-state image pickup elementaccording to the fourth embodiment is similar to the structure of thesolid-state image pickup element according to the first embodimentdescribed above with reference to FIG. 2A. In the fourth embodiment, aninterpolation processor 1301 (see FIG. 13) is formed on the secondsemiconductor chip 21.

FIG. 13 is a circuit diagram for illustrating the solid-state imagepickup element according to the fourth embodiment. As illustrated inFIG. 13, the plurality of pixels 201 arrayed in a two-dimensional matrixmanner are formed on the first semiconductor chip 20. Each of the pixels201 is connected to the transfer signal line 203, the reset signal line204, and the row selection signal line 205 in the horizontal direction,and is connected to the column signal line 202 a or the column signalline 202 b in the vertical direction. The pixel 201 connected to thecolumn signal line 202 a or 202 b differs depending on a readout row.That is, the pixel 201 located in the row including the pixels 201SHAand 201SHB (see FIG. 4A) for phase difference detection (for focusdetection), that is, the phase difference AF row, is connected to thecolumn signal line 202 b. Meanwhile, the pixel 201 located in the rownot including the pixels 201SHA and 201SHB for phase differencedetection, that is, the normal row, is connected to the column signalline 202 a. The configuration of the pixel 201 is similar to theconfiguration of the pixel 201 of the first embodiment described abovewith reference to FIG. 2C.

The second semiconductor chip 21 includes the AD conversion units 211.Each of the AD conversion units 211 is connected to the column signalline 202 a or the column signal line 202 b. The second semiconductorchip 21 further includes the row scanning circuit 212 that scansrespective rows, and the column scanning circuit 213 a and 213 b thatscan respective columns. The second semiconductor chip 21 furtherincludes the timing control circuit 214 that controls the operationtiming of each of the row scanning circuit 212, the column scanningcircuits 213 a and 213 b, and the AD conversion units 211 based on thecontrol signal output from the controller 106. The second semiconductorchip 21 further includes the horizontal signal lines 215 a and 215 b fortransferring the signals from the AD conversion units 211 in accordancewith the timing controlled by the column scanning circuits 213 a and 213b.

The second semiconductor chip 21 further includes the switch 216 thatswitches the output destination of the digital image signals output viathe horizontal signal line 215 b. When the switch 216 has the firstsetting, the output destination of the digital image signals output viathe horizontal signal line 215 b is set to the phase differenceevaluation value calculator 217 of the AF evaluation value calculator1021. Meanwhile, when the switch 216 has the second setting, the outputdestination of the digital image signals output via the horizontalsignal line 215 b is set to the interpolation processor 1301.

The pixel 201 located in the row including the pixels 201SHA and 201SHBfor phase difference detection, that is, the phase difference AF row, isconnected to the column signal line 202 b. Therefore, the signalsacquired by the pixels 201 located in the phase difference AF row aretransferred via the column signal lines 202 b and the horizontal signalline 215 b. The signals acquired by the pixels 201SHA or 201SHB forphase difference detection, which are located in the phase difference AFrow, are transferred to the phase difference evaluation value calculator217 via the column signal lines 202 b, the horizontal signal line 215 b,and the switch 216.

Incidentally, in the phase difference AF row, not only the pixels 201SHAand 201SHB for phase difference detection but also the pixels 201G forimage pickup are located (see FIG. 4A). When the pixels having specificspectral sensitivity like red (R), green (G), and blue (B) aredescribed, the reference symbols of 201R, 201G, and 201B are used,respectively. Further, when the pixels for the phase differencedetection are described, the reference symbols of 201SHA and 201SHB areused. Further, when general pixels are described, the reference symbolof 201 is used. The signal acquired by the pixel 201G for image pickupamong the pixels 201 located in the phase difference AF row istransmitted to the interpolation processor 1301 via the column signalline 202 b, the horizontal signal line 215 b, and the switch 216.

As described above, the pixel 201 located in the row not including thepixels 201SHA and 201SHB for phase difference detection, that is, thenormal row, is connected to the column signal line 202 a. Therefore, thesignal acquired by the pixel 201 located in the normal row is output tothe interpolation processor 1301 via the column signal line 202 a andthe horizontal signal line 215 a. As described above, the interpolationprocessor 1301 receives input of the digital image signal output via thehorizontal signal line 215 a and input of the digital image signaloutput via the horizontal signal line 215 b. The signal appropriatelysubjected to interpolation processing by the interpolation processor1301, that is, the image signal, is output to the contrast evaluationvalue calculator 218 of the AF evaluation value calculator 1021, and tothe signal processor 104.

The phase difference evaluation value calculated by the phase differenceevaluation value calculator 217 and the contrast evaluation valuecalculated by the contrast evaluation value calculator 218 are output tothe controller 106. The signal appropriately subjected to interpolationprocessing by the interpolation processor 1301, that is, the imagesignal, is transmitted to the signal processor 104 to be used for animage for live view at the time of live view display, for example.

A signal transferred to the horizontal signal line 215 a from the pixel201 located in the normal row is referred to as “first image signal”.Further, a signal transferred to the horizontal signal line 215 b fromthe pixel 201 located in the phase difference AF row is referred to as“second image signal”.

The rows selected at the time of readout in the solid-state image pickupelement according to the fourth embodiment are the same as the rowsselected at the time of readout in the solid-state image pickup elementaccording to the first embodiment, which are described with reference toFIG. 5. Also in the fourth embodiment, in order that acquisition of thefirst image signals and acquisition of the second image signals can beperformed in parallel, rows selected when the first image signals are tobe acquired and rows selected when the second image signals are to beacquired are each set. The first image signals are output to the columnsignal line 202 a. Meanwhile, the second image signals are output to thecolumn signal line 202 b.

FIG. 14 is a diagram for illustrating a layout of pixels in thesolid-state image pickup element according to the fourth embodiment.FIG. 14 is an illustration of a part of pixels 201 extracted from theplurality of pixels 201 arranged in matrix. The signals R, Gr, Gb, and Boutput from the pixels 201 located in a part of the plurality of normalrows, and the signals Gr and Gb output from the pixels 201G located in apart of the plurality of phase difference AF rows are also used when thecontrast evaluation value is acquired. The row in which the pixels 201that output signals to be used for acquiring the contrast evaluationvalue are located is also referred to as “contrast AF row” Asillustrated in FIG. 14, the contrast AF row includes the phasedifference AF rows and the normal rows. The contrast AF row is set so asto include not only the normal rows but also the phase difference AFrows because the contrast evaluation value can be acquired with highaccuracy by setting the contrast AF row to have a wide range.

The second image signals include signals acquired by the pixels 201SHAand 201SHB for phase difference detection, and hence increasing theframe rate becomes important to achieve a fast autofocus. Therefore, thesecond image signals are set to have a relatively high decimation ratio.Meanwhile, the first image signals are used for live view display andthe like, and hence the image quality becomes important. Therefore, thefirst image signals are set to have a relatively low decimation ratio.Focusing on the rows having the row numbers of 1 to 8, the first imagesignals are obtained by decimating one pixel among the four same-colorpixels arranged in the vertical direction, and the second image signalsare obtained by decimating three pixels among the four same-color pixelsarranged in the vertical direction. When the first image signals are tobe acquired, the first pixel group is read out at a first frame rate.When the second image signals are to be acquired, the second pixel groupis read out at a second frame rate that is higher than the first framerate. Description is given here of an example of a case in which thesecond frame rate is three times as high as the first frame rate.

As described above, in the first embodiment, the rows from which thefirst image signals are read out and the rows from which the secondimage signals are read out are separately set. Therefore, according tothe fourth embodiment, the first image signal and the second imagesignal having different charge accumulation time periods, different datasizes, and different frame rates can be acquired in parallel.Description has been given of an example of a case in which the secondpixel group for acquiring the second image signals is located in therows having the row numbers of 1 and 2, and the first pixel group foracquiring the first image signals is located in the rows having the rownumbers of 3 to 8, but the present invention is not limited thereto.Further, the decimation ratio for the readout can be set as appropriate.

The analog signal output from the pixel 201 to the column signal line202 a or the column signal line 202 b is converted from analog todigital in the AD conversion unit 211. The column scanning circuit 213 atransmits the digital first image signal output from the AD conversionunit 211 to the interpolation processor 1301 via the horizontal signalline 215 a. Further, the column scanning circuit 213 b transmits thedigital second image signal output from the AD conversion unit 211 tothe phase difference evaluation value calculator 217 or theinterpolation processor 1301 via the horizontal signal line 215 b.Further, the signal appropriately subjected to interpolation processingby the interpolation processor 1301, that is, the image signal, istransmitted to the contrast evaluation value calculator 218, and isoutput to the signal processor 104 via an output terminal 1302 of thesolid-state image pickup element 102.

The controller 106 uses the signal output from the phase differenceevaluation value calculator 217 of the solid-state image pickup element102 to perform autofocus control by the phase difference detectionmethod, that is, phase difference focus control (phase difference AF).Further, the controller 106 also uses the signal output from thecontrast evaluation value calculator 218 of the solid-state image pickupelement 102 to perform autofocus control by the contrast detectionmethod, that is, contrast focus control (contrast AF).

The phase difference evaluation value calculator 217 calculates thephase difference evaluation value for focus detection by the phasedifference detection method based on the pixels 201SHA and 201SHB forphase difference detection included in the plurality of pixels 201. Thephase difference evaluation value calculator 217 performs correlationcalculation with respect to a pair of image signals generated by thesignals SHA and SHB output from the plurality of pixels 201SHA and201SHB for phase difference detection, to thereby calculate a phasedifference representing a relative deviation between the pair of imagesignals. Then, the phase difference evaluation value calculator 217calculates a defocus amount Df, which represents an amount ofout-of-focus, based on the phase difference. The controller 106calculates, based on the defocus amount Df, the amount to move the focuslens in order to obtain a state close to an in-focus state, that is, afocus lens drive amount to a position near an in-focus position. Thecontroller 106 causes the drive unit 103 to control the opticalmechanism 1011 so that the focus lens is moved by the calculated driveamount. The contrast evaluation value calculator 218 calculates thecontrast evaluation value by extracting a high-frequency component inthe image signal output from the contrast AF row. The controller 106drives the focus lens as appropriate based on the contrast evaluationvalue.

The relationship between the image pickup optical system and the pixelsin the fourth embodiment is similar to the relationship between theimage pickup optical system and the pixels in the first embodimentdescribed above with reference to FIG. 3A to FIG. 4B.

An object image acquired by the plurality of pixels 201SHA for phasedifference detection regularly arranged in the x direction, that is, thefirst pixel group for phase difference detection, is referred to as“first image”. Further, an object image acquired by the plurality ofpixels 201SHB for phase difference detection regularly arranged in the xdirection, that is, the second pixel group for phase differencedetection, is referred to as “second image”. Further, a relativedeviation amount between the first image and the second image, that is,the phase difference, is detected so that the defocus amount Df of thefocus lens with respect to the object can be calculated based on thephase difference.

Signals output from the pixels 201 to the column signal lines 202 a and202 b are converted from analog to digital by the AD conversion units211. The signals converted into digital by the AD conversion units 211are output from the AD conversion units 211 to the horizontal signallines 215 a and 215 b by the column scanning circuits 213 a and 213 b,respectively. The signals R, Gr, Gb, and B output to the horizontalsignal line 215 a are output to the interpolation processor 1301.Meanwhile, the signal output to the horizontal signal line 215 b isoutput via the switch 216. When the switch 216 has the first setting,the signals SHA and SHB output from the pixels 201SHA and 201SHB forphase difference detection are input to the phase difference evaluationvalue calculator 217. When the switch 216 has the second setting, thesignals Gr and Gb output from the pixels 201G for image pickup, whichare located in the phase difference AF row, are input to theinterpolation processor 1301.

The phase difference evaluation value calculator 217 calculates thephase difference evaluation value through correlation calculation basedon the signals SHA and SHB transmitted to the phase differenceevaluation value calculator 217 via the horizontal signal line 215 b andthe switch 216. That is, the phase difference evaluation valuecalculator 217 calculates the phase difference evaluation value forfocus detection by the phase difference detection method based on thesignals SHA and SHB from the pixels 201SHA and 201SHB. The phasedifference evaluation value calculated by the phase differenceevaluation value calculator 217 is output to the controller 106 via theoutput terminal 219 of the solid-state image pickup element 102. Thephase difference evaluation value calculator 217 outputs the phasedifference evaluation value to the controller 106 as soon as thecalculation of the phase difference evaluation value is completedregardless of whether or not the output of the image signal acquired bythe pixel array 206 to the signal processor 104 is completed.

To the interpolation processor 1301, the signals R, Gr, Gb, and B readout from the normal rows are transmitted via the horizontal signal line215 a, and the signals Gr and Gb read out from the phase difference AFrow are transmitted via the horizontal signal line 215 b and the switch216. The interpolation processor 1301 uses those signals as appropriateto perform the interpolation processing.

FIG. 15A and FIG. 15B are diagrams for conceptually illustratinginterpolation processing. FIG. 15A is an illustration of a state inwhich the R signal that is absent at a position of the pixel 201SHA forphase difference detection is acquired by the interpolation processing.The interpolation processor 1301 generates a signal for compensating fora defect caused in the image signal acquired by the pixel array 206,that is, a defect caused because the pixel for phase differencedetection is included in the plurality of pixels, through interpolationwith use of signals from pixels other than the pixel for phasedifference detection. Specifically, the interpolation processor 1301uses the signals from the pixels 201 other than the pixel 201SHA forphase difference detection to generate the R signal that is absent atthe position of the pixel 201SHA for phase difference detection throughinterpolation processing. More specifically, the interpolation processor1301 acquires the R signal that is absent at the position of the pixel201SHA for phase difference detection by averaging signals from sixpixels 201R located around the pixel 201SHA for phase differencedetection.

Further, FIG. 15B is an illustration of a state in which the B signalthat is absent at a position of the pixel 201SHB for phase differencedetection is acquired by the interpolation processing. The interpolationprocessor 1301 uses the signals from the pixels 201 other than the pixel201SHB for phase difference detection to generate the B signal that isabsent at the position of the pixel 201SHB for phase differencedetection through interpolation processing. More specifically, theinterpolation processor 1301 acquires the B signal that is absent at theposition of the pixel 201SHB for phase difference detection by averagingsignals from six pixels 201B located around the pixel 201SHB for phasedifference detection. In general, the signals Gr and Gb transmitted viathe horizontal signal line 215 b are not subjected to special correctionprocessing. However, the signals Gr and Gb transmitted via thehorizontal signal line 215 b may be subjected to correction processingas appropriate based on the difference in characteristics between thesignals Gr and Gb transmitted via the horizontal signal line 215 b andthe signals Gr and Gb transmitted via the horizontal signal line 215 a.

The signals R and B corresponding to the positions of the pixels 201SHAand 201SHB for phase difference detection and being generated by theinterpolation processor 1301 through interpolation processing aretransmitted to the contrast evaluation value calculator 218 and thesignal processor 104. Further, the signals Gr and Gb output from thepixels 201G for image pickup, which are located in the phase differenceAF row, are transmitted to the contrast evaluation value calculator 218and the signal processor 104 via the interpolation processor 1301.Further, the signals R, Gr, Gb, and B output from the pixels 201 forimage pickup, which are located in the normal row, are transmitted tothe contrast evaluation value calculator 218 via the interpolationprocessor 1301.

The contrast evaluation value calculator 218 calculates the contrastevaluation value for focus detection by the contrast detection methodbased on the image signal including the signal generated by theinterpolation processor 1301 through interpolation. The contrastevaluation value calculator 218 performs contrast calculation tocalculate the contrast evaluation value based on the image signalsubjected to the interpolation processing. The contrast evaluation valuecalculated by the contrast evaluation value calculator 218 is output tothe controller 106 via the output terminal 220 of the solid-state imagepickup element 102. The contrast evaluation value calculator 218 outputsthe contrast evaluation value to the controller 106 as soon as thecalculation of the contrast evaluation value is completed regardless ofwhether or not the output of the image signal acquired by the pixelarray 206 to the signal processor 104 is completed.

In the fourth embodiment, the defects of the R and B signals caused inthe pixels 201SHA and 201SHB for phase difference detection areeliminated through the interpolation processing performed by theinterpolation processor 1301. Therefore, in the fourth embodiment, thecontrast evaluation value may be calculated based on signals from a widerange including the phase difference AF rows. The contrast evaluationvalue is calculated based on the signals from a wide range including thephase difference AF rows, and hence, according to the fourth embodiment,the contrast evaluation value can be obtained with high accuracy.

Further, in the fourth embodiment, the defects are eliminated by theinterpolation processor 1301, and hence the first image signals and thesecond image signals whose defects are eliminated may be used to obtaina highly-accurate image signal without decimation. The image signal isused for an image for live view at the time of live view display, forexample.

In this case, a path through which the first image signal is output viathe column signal line 202 a and the horizontal signal line 215 a isreferred to as “first channel CH1”, and a path through which the secondimage signal is output via the column signal line 202 b and thehorizontal signal line 215 b is referred to as “second channel CH2”.

FIG. 16 is a time chart for illustrating an operation of the imagepickup apparatus according to the fourth embodiment. FIG. 16 is anillustration of an operation in a mode in which live view display isperformed while an autofocus operation is performed, that is, an AFevaluation mode.

The AF evaluation mode is started when an AF control signal from thecontroller 106 is turned on. A timing at which the AF control signal isturned on is set to T0. At the timing T0, a vertical synchronizationsignal VD changes from a High level to a Low level. In synchronizationwith the vertical synchronization signal VD, acquisition of the firstimage signals, that is, acquisition of the signals from the pixels 201located in the normal row, and acquisition of the second image signals,that is, acquisition of the signals from the pixels 201 located in thephase difference AF row, are started. As described above, the firstimage signals are output via the first channel CH1, and the second imagesignals are output via the second channel CH2.

During a period between T0 and TF1, the signals SHA and SHB are read outfrom the pixels 201SHA and 201SHB located in the phase difference AFrow. The readout of the signals from the pixels 201SHA and 201SHBlocated in the phase difference AF row is performed in parallel to thereadout of the signals from the pixels 201 located in the normal row.The signals SHA and SHB for phase difference evaluation, which are readout from the pixels 201SHA and 201SHB located in the phase difference AFrow, are transmitted to the phase difference evaluation value calculator217 via the horizontal signal line 215 b and the switch 216. Then,during a period between TF1 and TF2, the phase difference evaluationvalue calculator 217 calculates the phase difference evaluation value.During a period between TF2 and TF3, the phase difference evaluationvalue calculated by the phase difference evaluation value calculator 217is output to the controller 106.

During a period between T0 and TF1C, the acquisition of the imagesignal, that is, the acquisition of the signals for live view, isperformed. The first image signals and the second image signals whosedefects are eliminated construct the image signal. During a periodbetween TF1C and TF2C, the contrast evaluation value calculator 218calculates the contrast evaluation value based on the signal subjectedto the interpolation processing by the interpolation processor 1301.During a period between TF2C and TF3C, the contrast evaluation valuecalculated by the contrast evaluation value calculator 218 is output tothe controller 106.

The vertical synchronization signal VD changes to the Low level at eachpredetermined time interval ΔT. The vertical synchronization signal thathas changed to the Low level returns to the High level after an elapseof a predetermined time period. The predetermined time interval ΔTcorresponds to a sum of a period required for acquisition of the signalsfor live view of one frame, a period required for calculating thecontrast evaluation value, and a period required for outputting thecontrast evaluation value. That is, the predetermined time interval ΔTcorresponds to a sum of the one-frame live view signal acquisitionperiod, a contrast evaluation value calculation period, and a contrastevaluation value outputting period. In the fourth embodiment, forexample, the AF evaluation is performed three times within thepredetermined time interval ΔT. Acquisition of the signals for phasedifference evaluation, that is, acquisition of the second image signals,is performed independently and in parallel to acquisition of the signalsfor live view, that is, acquisition of the first image signals.

The controller 106 determines whether or not a desired in-focus statecan be obtained. When the desired in-focus state can be obtained, the AFcontrol signal is turned off, that is, returned to the Low level. InFIG. 16, a timing T1 indicates the timing at which the AF control signalis turned off. When the AF control signal is turned off, the AFevaluation is canceled, but the image for live view is continuouslyacquired.

FIG. 17 is a flow chart for illustrating the operation of the imagepickup apparatus according to the fourth embodiment.

When the user operates the operation unit 108 to turn on the power ofthe image pickup apparatus 100, the controller 106 sets the image pickupapparatus 100 to a standby state. The controller 106 determines whetheror not the image pickup apparatus 100 is required to be operated in theAF evaluation mode based on the operation performed by the user on theoperation unit 108 or the like (Step S1701). When the image pickupapparatus 100 is not required to be operated in the AF evaluation mode(NO in Step S1701), acquisition of the signals for live view is started(Step S1702). Then, a live view image obtained by the acquisition of thesignals for live view is displayed on the image display 109 (StepS1720). On the other hand, when the image pickup apparatus 100 isrequired to be operated in the AF evaluation mode (YES in Step S1701),the controller 106 turns on the AF control signal (Step S1703), and setsthe number of times n of acquiring the signals for phase differenceevaluation to 0 (Step S1704).

After that, the controller 106 causes the solid-state image pickupelement 102 to start the acquisition of the signals for live view (StepS1712), and causes the solid-state image pickup element 102 to start theacquisition of the signals for phase difference evaluation (Step S1705).After the controller 106 causes the solid-state image pickup element 102to start the acquisition of the signals for phase difference evaluation,the controller 106 increments the number of times n of acquiring thesignals for phase difference evaluation (Step S1706). After that, thephase difference evaluation value is calculated by the phase differenceevaluation value calculator 217 (Step S1707). In the calculation of thephase difference evaluation value, the phase difference evaluation valuecalculator 217 calculates the phase difference evaluation value based onthe signals SHA and SHB for phase difference evaluation acquired in StepS1705, to thereby obtain the defocus amount Df. The phase differenceevaluation value calculator 217 outputs the phase difference evaluationvalue, specifically, the defocus amount Df, to the controller 106 assoon as the calculation of the defocus amount Df is completed. Afterthat, the processing proceeds to Step S1708.

In parallel to the operation of Step S1707, Step S1713 is performed. InStep S1713, the interpolation processor 1301 performs the interpolationprocessing. Then, after completion of the readout of the signals forlive view is waited for, the contrast evaluation value is calculated bythe contrast evaluation value calculator 218 (Step S1714). In thecalculation of the contrast evaluation value, the contrast evaluationvalue calculator 218 calculates the contrast evaluation value AF_K basedon the signals read out from the contrast AF row, that is, the signalsthat are subjected to interpolation processing as appropriate. Thecontrast evaluation value calculator 218 outputs the contrast evaluationvalue AF_K to the controller 106 as soon as the calculation of thecontrast evaluation value AF_K is completed. The contrast evaluationvalue AF_K is used in Step S1715.

In Step S1708, the controller 106 determines whether or not the defocusamount Df calculated in Step S1707 falls within a range of a desireddefocus amount based on Expression (1).

When the defocus amount Df does not satisfy Expression (1) (NO in StepS1708), the processing proceeds to Step S1709. In Step S1709, thecontroller 106 determines the feed-back control amount, that is, thefocus lens drive amount, based on the defocus amount Df. Then, thecontroller 106 causes the drive unit 103 to control the opticalmechanism 1011, to thereby drive the focus lens in the image pickupoptical system 101 (phase difference AF control). In Step S1710, thecontroller 106 determines whether or not the number of times n ofacquiring the signals for phase difference evaluation is 3. When thenumber of times n of acquiring the signals for phase differenceevaluation is not 3, the processing returns to Step S1705. On the otherhand, when the number of times n of acquiring the signals for phasedifference evaluation is 3, live view display is performed (Step S1717),and the processing returns to Step S1704.

When the defocus amount Df satisfies Expression (1) (YES in Step S1708),the processing proceeds to Step S1711. In Step S1711, the controller 106determines whether or not the number of times n of acquiring the signalsfor phase difference evaluation is 3. When the number of times n ofacquiring the signals for phase difference evaluation is not 3 (NO inStep S1711), the processing returns to Step S1705. On the other hand,when the number of times n of acquiring the signals for phase differenceevaluation is 3 (YES in Step S1711), the processing proceeds to StepS1715.

In Step S1715, the controller 106 determines whether or not the contrastevaluation value AF_K obtained in Step S1714 falls within a range of adesired contrast amount based on Expression (2).

When the contrast evaluation value AF_K does not satisfy Expression (2)(NO in Step S1715), the processing proceeds to Step S1716. In StepS1716, the controller 106 determines the feed-back control amount, thatis, the focus lens drive amount, based on the contrast evaluation valueAF_K. Then, the controller 106 causes the drive unit 103 to control theoptical mechanism 1011, to thereby drive the focus lens in the imagepickup optical system 101 (contrast AF control). After that, theprocessing proceeds to Step S1717. When the contrast evaluation valueAF_K satisfies Expression (2) (YES in Step S1715), the controller 106turns off the AF control signal (Step S1718). After that, the controller106 ends the acquisition of the signals for phase difference evaluation(Step S1719). The controller 106 displays the signals for live viewacquired in Step S1712 on the image display 109 (Step S1720).

As described above, according to the fourth embodiment, the defectscaused in the image signal because the pixel array 206 includes thepixels 201SHA and 201SHB for phase difference detection are compensatedfor with use of the signals from the pixels 201 other than the pixels201SHA and 201SHB for phase difference detection. Then, the contrastevaluation value for focus detection by the contrast detection method iscalculated based on the image signal subjected to interpolation. Thecontrast evaluation value may be calculated based on the image signal ina wide range including the rows including the pixels 201SHA and 201SHBfor phase difference detection. Therefore, according to the fourthembodiment, the contrast evaluation value can be obtained with highaccuracy. That is, according to the fourth embodiment, the contrastevaluation value can be obtained with high accuracy even when theplurality of pixels 201 constructing the pixel array 206 include thepixels 201SHA and 201SHB for phase difference detection. A fastautofocus can be achieved due to the pixels 201SHA and 201SHB for phasedifference detection, and hence according to the fourth embodiment, itis possible to provide the image pickup apparatus capable of achieving afast autofocus with high accuracy.

Fifth Embodiment

A solid-state image pickup element, a method of controlling asolid-state image pickup element, and an image pickup apparatusaccording to a fifth embodiment are described with reference to FIG. 18and FIG. 19. Like components in the solid-state image pickup element,the method of controlling a solid-state image pickup element, and theimage pickup apparatus according to the first to fourth embodimentsillustrated in FIG. 1 to FIG. 11D and FIG. 13 to FIG. 17 are denoted bylike reference symbols to omit or simplify the description.

The image pickup apparatus 100 according to the fifth embodimentcontinuously acquires still images, that is, perform continuous shootingof still images.

FIG. 18 is a time chart for illustrating an operation of the imagepickup apparatus according to the fifth embodiment. The still imagecontinuous shooting mode, which is a mode of continuously acquiring thestill images, is started when the still image continuous shootingcontrol signal from the controller 106 is turned on. The controller 106causes the drive unit 103 to control the mechanical shutter mechanismand the diaphragm mechanism of the optical mechanism 1011, and causesthe drive unit 103 to control the solid-state image pickup element 102.Specifically, the reset operation is performed on the pixels 201, andthe mechanical shutter (shutter) is opened so that exposure to thepixels 201 is started. After the exposure to the pixels 201 is started,photoelectric conversion by the photoelectric converters PD is started.

After an elapse of such a predetermined exposure time period thatsatisfies an exposure condition set in advance, the controller 106causes the drive unit 103 to close the shutter. When the shutter isclosed, the exposure to the pixels 201 is completed. After the exposureto the pixels 201 is completed, acquisition of the first image signals,that is, acquisition of the signals for the still image, and acquisitionof the second image signals, that is, acquisition of the signals forphase difference evaluation, are started. The timing at which theacquisition of the first image signals and the acquisition of the secondimage signals are started is TF0. The first image signals are output viathe first channel CH1, and the second image signals are output via thesecond channel CH2. The rows selected when the first image signals areto be acquired and the rows selected when the second image signals areto be acquired may be the same as those in the case of the firstembodiment described above with reference to FIG. 5, for example.

During a period between TF0 and TF1, the signals SHA and SHB for phasedifference evaluation are transmitted to the phase difference evaluationvalue calculator 217 via the horizontal signal line 215 b and the switch216. Then, during a period between TF1 and TF2, the phase differenceevaluation value calculator 217 calculates the phase differenceevaluation value. During a period between TF2 and TF3, the phasedifference evaluation value calculated by the phase differenceevaluation value calculator 217 is output to the controller 106. Thecontroller 106 causes the drive unit 103 to drive the focus lens of theimage pickup optical system 101 based on the phase difference evaluationvalue.

During a period between TF0 and TF1C, the signals for the still imageare acquired. During a period between TF1C and TF2C, the contrastevaluation value calculator 218 calculates the contrast evaluationvalue. During a period between TF2C and TF3C, the contrast evaluationvalue calculated by the contrast evaluation value calculator 218 isoutput to the controller 106. The controller 106 causes the drive unit103 to drive the focus lens of the image pickup optical system 101 basedon and the contrast evaluation value. When the drive of the focus lensis completed, and the still image continuous shooting mode is not ended,the processing proceeds to the acquisition of the next still image.

FIG. 19 is a flow chart for illustrating the operation of the imagepickup apparatus according to the fifth embodiment. First, thecontroller 106 causes the image pickup apparatus 100 to shift from thestandby state to the still image continuous shooting mode based on theoperation input performed by the user on the operation unit 108. In StepS1901, the controller 106 causes the drive unit 103 to open themechanical shutter of the optical mechanism 1011. In this manner, theexposure to the pixels 201 is started, and the photoelectric conversionby the photoelectric converters PD is started so that charges arestarted to be accumulated (Step S1902). After an elapse of apredetermined exposure time period, the controller 106 causes the driveunit 103 to close the mechanical shutter of the optical mechanism 1011(Step S1903).

Next, the controller 106 causes the solid-state image pickup element 102to start the acquisition of the signals for the still image (StepS1907), and causes the solid-state image pickup element 102 to start theacquisition of the signals for phase difference evaluation (Step S1904).After that, the phase difference evaluation value is calculated by thephase difference evaluation value calculator 217 (Step S1905). In thecalculation of the phase difference evaluation value (Step S1905), thephase difference evaluation value calculator 217 calculates the phasedifference evaluation value based on the signals SHA and SHB for phasedifference evaluation acquired in Step S1904, to thereby obtain thedefocus amount Df. The phase difference evaluation value calculator 217outputs the phase difference evaluation value, specifically, the defocusamount Df, to the controller 106 as soon as the calculation of the phasedifference evaluation value is completed. After that, the processingproceeds to Step S1906.

In Step S1906, the controller 106 determines whether or not the defocusamount Df calculated in Step S1905 falls within a range of a desireddefocus amount based on Expression (1) described above in the firstembodiment. When the defocus amount Df does not satisfy Expression (1)(NO in Step S1906), the processing proceeds to Step S1910. In StepS1910, the controller 106 determines the feed-back control amount, thatis, the focus lens drive amount, based on the defocus amount Df. In StepS1910, the controller 106 causes the drive unit 103 to control theoptical mechanism 1011, to thereby drive the focus lens in the imagepickup optical system 101 (phase difference AF control). After that, theprocessing proceeds to Step S1913. When the defocus amount Df satisfiesExpression (1) (YES in Step S1906), the processing proceeds to StepS1911.

In parallel to the operation of Step S1905, Step S1908 is performed. InStep S1908, the interpolation processor 1301 performs the interpolationprocessing. Then, after completion of the readout of the signals forlive view is waited for, the contrast evaluation value is calculated bythe contrast evaluation value calculator 218 (Step S1909). In thecalculation of the contrast evaluation value, the contrast evaluationvalue calculator 218 calculates the contrast evaluation value AF_K basedon the signal read out from the contrast AF row, that is, the signalthat is subjected to interpolation processing as appropriate. Thecontrast evaluation value calculator 218 outputs the contrast evaluationvalue AF_K to the controller 106 as soon as the calculation of thecontrast evaluation value AF_K is completed. The contrast evaluationvalue AF_K is used in Step S1911.

In Step S1911, the controller 106 determines whether or not the contrastevaluation value AF_K obtained in Step S1909 falls within a range of adesired contrast amount based on Expression (2) described above in thefirst embodiment. When the contrast evaluation value AF_K does notsatisfy Expression (2) (NO in Step S1911), the processing proceeds toStep S1912. In Step S1912, the controller 106 determines the feed-backcontrol amount, that is, the focus lens drive amount, based on thecontrast evaluation value AF_K. The controller 106 causes the drive unit103 to control the optical mechanism 1011, to thereby drive the focuslens in the image pickup optical system 101 (contrast AF control). Afterthat, the processing proceeds to Step S1913. When the contrastevaluation value AF_K satisfies Expression (2) (YES in Step S1911), theprocessing proceeds to Step S1913.

In Step S1913, the controller 106 determines whether or not to end thestill image continuous shooting mode in accordance with the operationinput performed by the user on the operation unit 108. When the stillimage continuous shooting mode is not to be ended (NO in Step S1913),the processing returns to Step S1901 to start the acquisition of thenext still image. On the other hand, when the still image continuousshooting mode is to be ended (YES in Step S1913), the controller 106causes the image pickup apparatus 100 to shift to the standby statewithout acquiring the next still image.

As described above, the present invention is applicable also to a casein which the still images are continuously taken. Also in the fifthembodiment, it is possible to provide the image pickup apparatus capableof achieving a fast autofocus with high accuracy.

FIG. 34 is a timing chart for illustrating an example of a hybrid AF inan image pickup apparatus according to Reference Example 2. In thehybrid AF, as described above, the focus detection is first performedwith use of the phase difference detection method, and then the focusdetection is performed with use of the contrast detection method. In thesolid-state image pickup element, a plurality of phase differencedetection pixels that each receive a pair of light fluxes that haspassed through different pupil partial regions of the image pickupoptical system are arranged in a part of the region of the pixel array.Now, the hybrid AF of Reference Example 2 is described with reference toFIG. 34.

For example, when the still image continuous shooting mode is starteddue to the user operation, the still image continuous shooting controlsignal representing the still image continuous shooting mode is set toHIGH. In this manner, at the same time as when the opening of themechanical shutter is opened, the charges accumulated in the pixels arereset, and the exposure to the pixels is started (time T0). Then, afteran elapse of a predetermined exposure period, the mechanical shutter isclosed to block the light reflected from the object. Then, the exposureto the pixels is ended (time TF0).

Next, readout from the normal image pickup pixels is started (time TF0).After the image pickup signals are read out from all of the image pickuppixels, the readout from the phase difference detection pixels issubsequently started. After the phase difference signals are read outfrom all of the phase difference detection pixels (time TF1), thesolid-state image pickup element calculates the phase differenceevaluation value based on the phase difference signals, and calculatesthe contrast evaluation value based on the image pickup signals (periodbetween TF1 and TF2).

Next, the solid-state image pickup element outputs, to a controllerprovided outside of the solid-state image pickup element, the calculatedphase difference evaluation value and contrast evaluation value in aform of being added to the image pickup signals (period between TF2 andTF3). The controller drives the focus lens based on the phase differenceevaluation value and the contrast evaluation value, to thereby performthe in-focus operation (period between TF3 and TF4). After the in-focusoperation is completed, as in the time T0, the operation of exposure tothe pixels is started to take the next still image (time TF4).

As described above, in the hybrid AF of Reference Example 2, after thecompletion of the readout of the image pickup signals for generating aphotographing image is waited for, the calculation of the phasedifference evaluation value and the contrast evaluation value isstarted, and hence there has been a problem in that the hybrid AFrequires time until completion. As a result, for example, the continuousshooting speed in the still image continuous shooting is decreased. Inembodiments described below, a method of increasing the speed of thehybrid AF is described.

Sixth Embodiment

An image pickup apparatus according to a sixth embodiment is describedwith reference to FIG. 20 to FIG. 30. The configuration of the imagepickup apparatus according to the sixth embodiment is similar to theconfiguration of the image pickup apparatus according to the firstembodiment described above with reference to FIG. 1. The image pickupapparatus 100 illustrated in FIG. 1 includes the image pickup opticalsystem 101, the solid-state image pickup element 102, the drive unit103, the signal processor 104, the compression/expansion unit 105, thecontroller 106, the light emission unit 107, the operation unit 108, theimage display 109, and the image recording unit 110. The image pickupapparatus according to the sixth embodiment is applied to a still cameraor a video camera, for example.

The image pickup optical system 101 includes an image pickup opticalsystem for condensing light reflected from an object to the solid-stateimage pickup element 102, a lens, and the optical mechanism 1011. Theoptical mechanism 1011 is driven by the drive unit 103 to adjust thezooming, the shutter, the aperture value, and the focusing of the imagepickup optical system 101. The solid-state image pickup element 102includes pixels arranged in matrix, and an AD converter that read outthe signals output from the pixels. The solid-state image pickup element102 performs operations such as exposure to each pixel, signal readout,and reset. As the solid-state image pickup element 102, for example, anXY-readout-type CMOS image sensor is used. The drive unit 103 iscontrolled by the controller 106 to drive the optical mechanism 1011 andthe solid-state image pickup element 102.

The AF evaluation value calculator 1021 in the solid-state image pickupelement 102 is controlled by the controller 106 to calculate the phasedifference evaluation value and the contrast evaluation value based onthe image pickup signals and the phase difference signals read out fromthe pixels, to output the values to the controller 106. The controller106 includes, for example, a central processor (CPU), a read only memory(ROM), and a random access memory (RAM). The controller 106 generallycontrols the image pickup apparatus by loading a program stored in theROM or the like to the RAM and executing the program by the CPU.

The signal processor 104 is controlled by the controller 106 to subjectthe image pickup signals output from the solid-state image pickupelement 102 to signal processing such as white balance adjustmentprocessing, color correction processing, or auto-exposure (AE)processing, to output the corrected signals to the compression/expansionunit 105. The compression/expansion unit 105 is controlled by thecontroller 106 to subject the corrected image pickup signals to encodingprocessing to output the encoded signals to the controller 106. As themethod of encoding the image pickup signals by the compression/expansionunit 105, for example, when a still image is to be generated, a jointphotographic coding experts group (JPEG) method is used. Further, when amoving image is to be generated, a moving picture experts group (MPEG)method is used. The compression/expansion unit 105 may have a functionof decoding in addition to the function of encoding.

The light emission unit 107 irradiates the object with light when thesignal processor 104 determines that the brightness of the object is lowby the AE processing, for example. As the light emission unit 107, forexample, a stroboscopic device using a xenon tube or an LED lightemitting device can be used. The operation unit 108 includes, forexample, various operation keys including a shutter release button, alever, and a dial. The operation unit 108 outputs a signal correspondingto the operation performed by the user on the controller 106. The imagedisplay 109 includes a display device, for example, a liquid crystaldisplay (LCD), and an interface circuit that drive the display device.The image display 109 displays the image data output from the controller106 on the display device. The image recording unit 110 is constructedwith use of, for example, a portable semiconductor memory, an opticaldisc, a hard disk drive (HDD), or a magnetic tape. The image recordingunit 110 records the encoded image data output from the controller 106.

Now, the flow of the signals and the data in the image pickup apparatusaccording to the sixth embodiment is briefly described. The image pickupsignals read out from the pixels of the solid-state image pickup element102 are subjected to correlated double sampling (CDS) processing,automatic gain control (AGC) processing, or the like to be digitalizedby the AD conversion unit.

The digitalized image pickup signals are output to the signal processor104 and the AF evaluation value calculator 1021 in the solid-state imagepickup element 102. The AF evaluation value calculator 1021 calculatesthe contrast evaluation value based on the contrast information of theimage pickup signals to output the contrast evaluation value to thecontroller 106. The signal processor 104 subjects the image pickupsignals to image quality correction processing to output the correctedsignals to the controller 106. The controller 106 displays the imagepickup signals as a live view image on the image display 109. As aresult, the user can adjust the angle of view or the like while viewingthe displayed live view image.

Next, when the shutter release button of the operation unit 108 isdepressed under a state in which the live view image is displayed on theimage display 109, the solid-state image pickup element 102 outputs theimage pickup signals to the signal processor 104. The signal processor104 subjects the image pickup signals to image quality correctionprocessing to output the corrected signals to the compression/expansionunit 105. The compression/expansion unit 105 encodes the corrected imagepickup signals to output the encoded signals to the controller 106. Thecontroller 106 records the encoded image pickup signals in the imagerecording unit 110 as a still image file or a moving image file.

Next, when a play button of the operation unit 108 is depressed under astate in which the still image file or the moving image file recorded inthe image recording unit 110 is selected, the controller 106 reads outthe selected still image file or moving image file from the imagerecording unit 110 to output the still image file or the moving imagefile to the compression/expansion unit 105. The compression/expansionunit 105 decodes the read image data to output the decoded image data tothe controller 106. The controller 106 displays the decoded image dataon the image display 109. As a result, the user can play and view therecorded still image or moving image.

Meanwhile, the AF evaluation value calculator 1021 performs correlationcalculation with respect to a pair of phase difference signals outputfrom the phase difference detection pixels, to thereby obtain a relativeimage deviation amount. Then, the defocus amount of the image pickupoptical system 101 is calculated (detected) based on the image deviationamount. The controller 106 calculates the drive amount for moving thefocus lens to a position close to the in-focus position based on thedefocus amount. Then, the drive unit 103 is controlled based on thedrive amount to drive the focus lens in the optical mechanism 1011.Thus, the in-focus operation is performed.

Further, while controlling the optical mechanism 1011 to move the focuslens, the controller 106 acquires the contrast evaluation value from theAF evaluation value calculator 1021 at a predetermined cycle. Then, thein-focus operation is performed by driving the focus lens in a directionin which the contrast evaluation value calculated based on the imagepickup signals of the still image taken at this time becomes larger thanthe contrast evaluation value calculated based on the image pickupsignals of the still image taken at a previous time.

FIG. 20 is a schematic diagram for illustrating a configuration of thesolid-state image pickup element 102 according to the sixth embodiment.As illustrated in FIG. 2A, the solid-state image pickup element 102according to the sixth embodiment includes the first semiconductor chip20 and the second semiconductor chip 21 stacked on the firstsemiconductor chip 20.

The first semiconductor chip 20 includes the plurality of pixels 201arranged in matrix. Each of the plurality of pixels 201 is connected to,for each row, the transfer signal line 203, the reset signal line 204,and the row selection signal line 205, and is connected to, for eachcolumn, a column signal line 202.

Meanwhile, the second semiconductor chip 21 has arranged thereon the ADconversion units 211, the row scanning circuit 212, a column scanningcircuit 213, the timing control circuit 214, the switch 216, and the AFevaluation value calculator 1021. As described above, the pixels 201 andthe peripheral circuits therefor are arranged separately on the firstsemiconductor chip 20 and the second semiconductor chip 21. In thismanner, the peripheral circuits may have thinner wiring and higherdensity, and thus the solid-state image pickup element 102 can beincreased in speed, downsized, and sophisticated in function.

The AD conversion unit 211 subjects the signal output to the columnsignal line 202 to AD conversion to output the signal to the horizontalsignal line 215. The row scanning circuit 212 scans respective rows ofthe solid-state image pickup element 102, and the column scanningcircuit 213 scans respective columns of the solid-state image pickupelement 102. The timing control circuit 214 is controlled by thecontroller 106 to control the scanning timing of the row scanningcircuit 212 and the column scanning circuit 213.

The switch 216 distributes and supplys the digital signals output to thehorizontal signal line 215 to the phase difference evaluation valuecalculator 217, and the contrast evaluation value calculator 218 and thesignal processor 104. The AF evaluation value calculator 1021 includesthe phase difference evaluation value calculator 217 and the contrastevaluation value calculator 218. The AF evaluation value calculator 1021detects (calculates) the AF evaluation value to be used for focusdetection. The phase difference evaluation value calculator 217 and thecontrast evaluation value calculator 218 are described in detail later.

FIG. 21 is a schematic diagram for illustrating a configuration of thepixel 201 in the image pickup element 102 according to the sixthembodiment. The pixel 201 includes the photoelectric converter PD, thetransfer transistor M1, the reset transistor M2, the amplificationtransistor M3, and the selection transistor M4. As those transistors,for example, n-channel metal-oxide-semiconductor field-effecttransistors (MOSFETs) can be used.

The transfer signal line 203, the reset signal line 204, and the rowselection signal line 205 are connected to the gate terminal of thetransfer transistor M1, the gate terminal of the reset transistor M2,and the gate terminal of the selection transistor M4, respectively.Control signals are output from the row scanning circuit 212 to thosesignal lines so that the pixels 201 in the same row are simultaneouslydriven. With such a configuration, an operation of a row sequentialoperation rolling shutter or an operation of an all-row simultaneousdrive global shutter can be achieved. The column signal line 202 isconnected to the source terminal of the selection transistor M4.

The photoelectric converter PD accumulates charges generated throughphotoelectric conversion. A p-side of the photoelectric converter isgrounded, and an n-side thereof is connected to the source terminal ofthe transfer transistor M1. When the transfer transistor M1 is turnedon, the charges accumulated in the photoelectric converter PD aretransferred to the floating diffusion FD. In this case, the floatingdiffusion FD refers to a floating capacitance region formed at a node ofthree terminals, specifically, the drain terminal of the transfertransistor M1, the gate terminal of the amplification transistor M3, andthe source terminal of the reset transistor M2.

The amplification transistor M3 outputs a signal corresponding to theamount of charges transferred to the floating diffusion FD. The powersupply voltage Vdd is supplied to the drain terminal of theamplification transistor M3. The selection transistor M4 selects thepixel from which the signal is read out. The drain terminal of theselection transistor M4 is connected to the source terminal of theamplification transistor M3, and the source terminal of the selectiontransistor M4 is connected to the column signal line 202. When theselection transistor M4 is turned on, a signal corresponding to theamount of charges transferred to the floating diffusion FD is output tothe column signal line 202. The reset transistor M2 resets the chargestransferred to the floating diffusion FD. The power supply voltage Vddis supplied to the drain terminal of the reset transistor M2, and thesource terminal of the reset transistor M2 is connected to the floatingdiffusion FD.

FIG. 22A and FIG. 22B are schematic diagrams for illustrating astructure of image pickup pixels of the image pickup element 102according to the sixth embodiment. In this case, the image pickup pixelrefers to a normal pixel 201 not including a sub-pixel for focusdetection. FIG. 22A is a plan view of the image pickup pixels, and FIG.22B is a sectional view taken along the line A-A of FIG. 22A. Forexample, in the solid-state image pickup element 102 employing the Bayerarrangement, as illustrated in FIG. 22A, pixels having spectralsensitivity of green (G) are arranged at two diagonal locations in thepixels of two rows and two columns, and pixels having spectralsensitivity of red (R) and blue (B) are arranged at the two remaininglocations.

The image pickup pixels illustrated in FIG. 22B include the microlensesML, the color filters CF, the wiring layer CL, and the photoelectricconverters PD. In this case, the suffix of the color filter CFrepresents the color of the light that is allowed to pass therethrough.For example, the color filter CF_(R) allows red light (Red) to passtherethrough, and the color filter CF_(G) allows green light (Green) topass therethrough. Similarly, the color filter CF_(B) (not shown) allowsblue light (Blue) to pass therethrough. The photoelectric converter PDcorresponds to the photoelectric converter PD of the CMOS sensorillustrated in FIG. 21. The wiring layer CL corresponds to the signallines for transmitting various signals in the CMOS sensor.

The microlens ML and the photoelectric converter PD of the image pickuppixel are configured so that the light flux that has passed through theimage pickup optical system TL can be introduced as effectively aspossible. In other words, the exit pupil EP of the image pickup opticalsystem TL and the photoelectric converter PD are arranged in a conjugaterelationship due to the microlens ML. As a result, the exit pupil EPcorresponding to the photoelectric converter PD has a large diameter,and the effective area of the photoelectric converter PD is increased.Thus, the light flux from the object can be efficiently introduced toimprove the S/N ratio. FIG. 22B is an illustration of the structure ofthe R pixel, but the G pixel and the B pixel also have the samestructure.

FIG. 23A and FIG. 23B are schematic diagrams for illustrating thestructure of the phase difference detection pixel in the solid-stateimage pickup element 102 according to the sixth embodiment. In thiscase, the phase difference detection pixel refers to the pixel 201including the sub-pixel for focus detection. FIG. 23A is a plan view ofthe phase difference detection pixel, and FIG. 23B is a sectional viewtaken along the line B-B of FIG. 23A.

Among the RGB signals output from the RGB pixels, the G signal is a maincomponent of the brightness information. Human image recognitioncharacteristics are sensitive to brightness information, and hence theimage quality deterioration is easily recognized when the G signal isdefective. Meanwhile, the R signal or the B signal is a signal foracquiring color information, but humans are insensitive to colorinformation, and hence the image quality deterioration is less noticedeven when the R signal or the B signal is slightly defective. In view ofthis, in the sixth embodiment, as illustrated in FIG. 23A, two G pixelsof the four pixels arranged in two rows and two columns are left asnormal image pickup pixels, and sub-pixels SHA and SHB are arranged at apredetermined ratio at locations of the R pixel and the B pixel.

In the sixth embodiment, the sub-pixel SHA and the sub-pixel SHBconstruct the phase difference detection pixel. The microlens ML and thephotoelectric converter PD have substantially the same structure asthose of the image pickup pixel illustrated in FIG. 22B. In the sixthembodiment, the phase difference signal output from the sub-pixel is notused as the image pickup signal for generating the photographing image.Therefore, as illustrated in FIG. 23B, a color filter CF_(W) being atransparent film (white film) is arranged instead of the color filterfor color separation. The phase difference detection pixel can also beused as the image pickup pixel. In this case, a sum of the pair of phasedifference signals output from the phase difference detection pixels isused as the image pickup signal.

In the phase difference detection pixel illustrated in FIG. 23B, pupildivision is performed in the x direction. The opening portion OP_(HA) ofthe wiring layer CL of the sub-pixel SHA is deviated to the right side(−x direction), and the photoelectric converter PD of the sub-pixel SHAreceives a light flux that has passed through the exit pupil region(first pupil region) EP_(HA) on the left side (+x direction) of theimage pickup optical system TL. Meanwhile, the opening portion OP_(HB)of the wiring layer CL of the sub-pixel SHB is deviated to the left side(+x direction), and the photoelectric converter PD of the sub-pixel SHBreceives a light flux that has passed through the exit pupil region(second pupil region) EP_(HB) on the right side (−x direction) of theimage pickup optical system TL.

An object image acquired by the plurality of sub-pixels SHA (first pixelgroup) regularly arranged in the x direction is referred to as “image A”(first image). Further, an object image acquired by the plurality ofsub-pixels SHB (second pixel group) regularly arranged in the xdirection is referred to as “image B” (second image). The relativedeviation amount (phase difference) between the image A and the image Bis detected so that the defocus amount of the image pickup opticalsystem TL with respect to the object can be calculated.

Now, referring back to FIG. 20, the operation of the solid-state imagepickup element 102 is described. The image pickup signals and the phasedifference signals that are output to the column signal line 202 aredigitalized by the AD conversion unit 211. The digitalized phasedifference signals and image pickup signals are output to the switch 216via the horizontal signal line 215. The switch 216 switches the outputdestination of the signals read out via the column signal line 202. Withthe switch 216, the phase difference signals are output to the phasedifference evaluation value calculator 217, and the image pickup signalsare output to the contrast evaluation value calculator 218 and thesignal processor 104.

The phase difference evaluation value calculator 217 performscorrelation calculation based on the phase difference signals tocalculate the phase difference evaluation value. Meanwhile, the contrastevaluation value calculator 218 performs contrast calculation based onthe image pickup signals to calculate the contrast evaluation value. Thecalculated phase difference evaluation value and contrast evaluationvalue are output to the controller 106 via the signal processor 104 in aform of being added to the image pickup signals at a timing at which thereadout of the image pickup signals from the image pickup pixels iscompleted.

Next, with reference to FIG. 24 to FIG. 27B, focus detection by thephase difference method is described. FIG. 24 is a schematic diagram forillustrating a focus detection region 602 on a pixel array 601 in thesolid-state image pickup element 102 according to the sixth embodiment.The focus detection region 602 illustrated in FIG. 24 represents a pixelregion in which focus detection can be performed by the phase differencemethod. Further, shift regions 603 on both sides of the focus detectionregion 602 are regions necessary for correlation calculation. That is,for focus detection by the phase difference method, a pixel region 604including the focus detection region 602 and the shift regions 603 isnecessary. Symbols p, q, s, and t of FIG. 24 represent coordinates inthe horizontal direction (x-axis direction). Among the symbols, p and qrepresent x coordinates at the start point and the end point of thepixel region 604, respectively, and s and t represent x coordinates atthe start point and the end point of the focus detection region 602,respectively.

FIG. 25A to FIG. 25C are schematic graphs for showing an example of apair of phase difference signals 701 and 702 for focus detection in theimage pickup element 102 according to the sixth embodiment. In FIG. 25A,one phase difference signal 701 of the pair of phase difference signalssubjected to filtering is indicated by the solid line, and the otherphase difference signal 702 is indicated by the broken line. The phasedifference signals 701 and 702 of FIG. 25A technically represent a pairof object images obtained along the x direction, which are obtained fromthe plurality of sub-pixels SHA and SHB arranged in the x direction. Inthe following description, such a pair of object images is referred toas “phase difference signals 701 and 702”.

When the correlation amount of the pair of phase difference signals 701and 702 is to be calculated, the phase difference signals 701 and 702are each shifted by one pixel in directions opposite to each other asshown in FIG. 25B and FIG. 25C. FIG. 25B is a graph for showing theresult of shifting the phase difference signals 701 and 702 shown inFIG. 25A in directions of the arrows shown in FIG. 25B. Meanwhile, FIG.25C is a graph for showing the result of shifting the phase differencesignals 701 and 702 shown in FIG. 25A in directions of the arrows shownin FIG. 25C, which are opposite to the arrows shown in FIG. 25B.

The phase difference evaluation value calculator 217 calculates, whileshifting the pair of phase difference signals 701 and 702 by one pixelas shown in FIG. 25B and FIG. 25C, a sum of the absolute value of thedifference between the phase difference signal 701 and the phasedifference signal 702 in the focus detection region 602. A correlationamount COR is obtained by Expression (3) with use of a shift amount i, aminimum shift amount p-s, a maximum shift amount q-t, a start coordinates of the focus detection region 602, and an end coordinate t of thefocus detection region 602.

$\begin{matrix}{{{{COR}\lbrack i\rbrack} = {\sum\limits_{x = s}^{t}{{{A\lbrack {x + i} \rbrack} - {B\lbrack {x - i} \rbrack}}}}}\{ {( {p - s} ) < i < ( {q - t} )} \}} & (3)\end{matrix}$

FIG. 26A and FIG. 26B are graphs for showing a relationship between theshift amount i and the correlation amount COR of the phase differencesignals in the image pickup element 102 according to the sixthembodiment. The lateral axis represents the shift amount i, and thevertical axis represents the correlation amount COR. The correlationamount COR shown in FIG. 26A changes in accordance with the shift amounti, and the change amount of the correlation amount COR is zero atextreme values 802 and 803. In FIG. 26B, the vicinity of the extremevalue 802, which is a smaller one of the extreme values 802 and 803, isshown in an enlarged manner. A correlation change amount ΔCOR per unitshift amount i is obtained by Expression (4) with use of the shiftamount i, the minimum shift amount p-s, the maximum shift amount q-t,the start coordinate s of the focus detection region 602, and the endcoordinate t of the focus detection region 602.

ΔCOR[i]=COR[i−1]−COR[i+1]

{(p−s+1)<(q−t−1)}  (4)

FIG. 27A and FIG. 27B are graphs for showing a relationship between theshift amount i and the correlation change amount ΔCOR of the phasedifference signals in the image pickup element 102 according to thesixth embodiment. The lateral axis represents the shift amount i, andthe vertical axis represents the correlation change amount ΔCOR per unitshift amount i. The correlation change amount ΔCOR shown in FIG. 27Achanges in accordance with the shift amount i, and the value changesfrom negative to positive at the extreme values 802 and 803. A point atwhich the correlation change amount ΔCOR becomes zero is calledzero-crossing, at which the pair of phase difference signals 701 and 702has the highest degree of coincidence. The shift amount i for obtainingthe zero-crossing corresponds to the image deviation amount.

The phase difference evaluation value calculator 217 outputs the shiftamount i for obtaining the zero-crossing to the controller 106 as thephase difference evaluation value for calculating the drive amount ofthe focus lens. FIG. 27B is a graph for showing the vicinity of theextreme value 802 for obtaining the zero-crossing in an enlarged manner.The shift amount i=α+β for obtaining the zero-crossing is divided intoan integer part β(=k−1) and a decimal part α. The decimal part α of theimage deviation amount is calculated by Expression (5) based on thesimilarity relationship between a triangle ABC and a triangle ADE shownin FIG. 27B.

AB:AD=BC:DE

ΔCOR[k−1]:ΔCOR[k−1]−ΔCOR[k]=α:k−(k−1)

α=ΔCOR[k−1]/(ΔCOR[k−1]−ΔCOR[k])  (5)

Further, the integer part 3 of the image deviation amount is calculatedby Expression (6).

β=k−1  (6)

As shown in FIG. 27A, when the correlation change amount ΔCOR has aplurality of zero-crossings, the shift amount i at the zero-crossingwhen the correlation change amount ΔCOR has the largest change ratio(hereinafter referred to as “steepness”) is output to the controller 106as the phase difference evaluation value. The steepness is an index thatrepresents the easiness of the focus detection, and the focus detectioncan be performed with higher accuracy as the steepness is increased. Asteepness maxder is obtained by Expression (7).

maxder=|ΔCOR[k−1]|+|ΔCOR[k]|  (7)

Next, a method of calculating the reliability of the phase differenceevaluation value is described. The evaluation value reliability isdefined by a degree of coincidence fnclvl of the pair of phasedifference signals 701 and 702. The degree of coincidence fnclvl is anindex that represents an accuracy of the image deviation amount, and thefocus detection can be performed with higher accuracy as the value isincreased. The degree of coincidence fnclvl is calculated by Expression(8) as a reciprocal of the magnitude of the extreme value 802 shown inFIG. 26B.

(a): when |ΔCOR[k−1]|×2≤maxder,

fnclvl=4/(COR[k−1]+ΔCOR[k−1])

(b): when |ΔCOR[k−1]|×2>maxder,

fnclvl=4/(COR[k]−ΔCOR[k])  (8)

Next, the hybrid AF in the image pickup apparatus according to the sixthembodiment is described. FIG. 28 is a schematic diagram for illustratingthe arrangement of the sub-pixels in the solid-state image pickupelement 102 according to the sixth embodiment. In the image pickupapparatus according to the sixth embodiment, the sub-pixels SHA and SHBthat output the phase difference signals 701 and 702 and the normalimage pickup pixels are arranged in matrix in the solid-state imagepickup element 102 as illustrated in FIG. 28, for example. In FIG. 28,the same pixel arrangement is repeated at an eight-row cycle, and ineach cycle of the pixel arrangement, the sub-pixels SHA and SHB and theimage pickup pixels are arranged in the first and second rows, and theimage pickup pixels are arranged in the third to eighth rows.

The row scanning circuit 212 of the sixth embodiment first scans thephase difference detection row in which the sub-pixels SHA and SHB arearranged, and then scans the image pickup row in which the sub-pixelsSHA and SHB are not arranged. Specifically, when the image pickup isstarted due to the operation on the operation unit 108 or the like,first, the first row, the second row, the ninth row, the tenth row, andso on, which are the phase difference detection rows, are sequentiallyscanned. After that, the third to eighth rows, the eleventh to sixteenthrows, and so on, which are the image pickup rows, are sequentiallyscanned. As described above, the row scanning is started from the phasedifference detection row in which the sub-pixels SHA and SHB arearranged. Thus, the phase difference signals 701 and 702 for focusdetection, which are output from the sub-pixels SHA and SHB, can beacquired without waiting for the completion of the readout of the imagepickup signals for generating the photographing image.

The pixel arrangement in the solid-state image pickup element 102 is notnecessarily limited to the eight-row cycle as illustrated in FIG. 28,and is not required to be cyclic. In the solid-state image pickupelement 102 according to the sixth embodiment, at least one phasedifference detection row in which the sub-pixels SHA and SHB arearranged and at least one image pickup row in which the sub-pixels SHAand SHB are not arranged are only required to be provided.

FIG. 29 is a timing chart for illustrating a method of controlling theimage pickup element 102 according to the sixth embodiment. As describedabove, in the hybrid AF of the sixth embodiment, first, the phasedifference detection row in which the sub-pixels SHA and SHB arearranged is scanned, and then the image pickup row in which only thenormal image pickup pixels are arranged is scanned.

For example, when the still image continuous shooting mode is starteddue to the operation performed by the user on the operation unit 108,the controller 106 sets the still image continuous shooting controlsignal representing the still image continuous shooting mode to HIGH(time T0). In this manner, at the same time as when the opening of themechanical shutter of the optical mechanism 1011 is opened, the chargesaccumulated in the pixels 201 are reset, and the exposure to the pixels201 is started. Then, after elapse of an exposure period set so as tosatisfy a predetermined exposure condition, the mechanical shutter ofthe optical mechanism 1011 is closed to block the light reflected fromthe object. Thus, the exposure to the pixels 201 is ended (time TF0).

The AF evaluation value calculator 1021 first performs focus detectionwith use of the phase difference detection method. The phase differenceevaluation value calculator 217 of the AF evaluation value calculator1021 starts readout of the phase difference signals in order tocalculate the phase difference evaluation value necessary for the phasedifference AF (time TF0). After the phase difference signals are readout from all of the phase difference detection pixels (time TF1), thereadout of the image pickup signals from the image pickup pixels issubsequently started. While the image pickup signals are read out fromthe image pickup pixels, the phase difference evaluation valuecalculator 217 calculates the phase difference evaluation value based onthe phase difference signals (period between TF1 and TF2). After that,the evaluation value reliability for determining whether or not focus tothe object can be achieved is calculated based on the calculated phasedifference evaluation value (period between TF2 and TF3).

When the calculated evaluation value reliability is equal to or largerthan a predetermined threshold value, a determination unit (not shown)of the solid-state image pickup element 102 determines that the focus tothe object can be achieved by the focus detection using the phasedifference detection method. Then, after the image pickup signals areread out from all of the image pickup pixels, the phase differenceevaluation values are output to the controller 106 via the signalprocessor 104 in a form of being added to the image pickup signals(period between TF4 and TF5). The controller 106 drives the focus lensof the optical mechanism 1011 based on the phase difference evaluationvalue, to thereby perform the in-focus operation (period between TF5 andTF7). After the in-focus operation is completed, as in the time T0, theoperation of exposure to the pixels 201 is started to take the nextstill image (time TF7).

On the other hand, when the calculated evaluation value reliability issmaller than the predetermined threshold value, the determination unit(not shown) of the solid-state image pickup element 102 determines thatit is difficult to achieve the focus to the object by only the focusdetection using the phase difference detection method. Thus, the AFevaluation value calculator 1021 subsequently performs focus detectionwith use of the contrast detection method. The determination unit (notshown) may be the phase difference evaluation value calculator 217, ormay be the contrast evaluation value calculator 218. Alternatively, thedetermination unit (not shown) may be a peripheral circuit (circuit)provided on the second semiconductor chip 21 illustrated in FIG. 20.

Now, description is briefly given of the contrast AF before theoperation of the contrast evaluation value calculator 218 of the AFevaluation value calculator 1021 is described. In the contrast AF, thecontrast evaluation value is calculated at a predetermined cycle whilethe optical mechanism 1011 is controlled to move the focus lens. Then,the in-focus operation is performed by driving the focus lens in adirection in which the contrast evaluation value calculated based on theimage pickup signals of the still image taken at this time becomeslarger than the contrast evaluation value calculated based on the imagepickup signals of the still image taken at a previous time. In order todetermine the direction to drive the focus lens in the contrast AF, thecontrast evaluation values are necessary at least at two differenttimes.

Therefore, when the first still image is taken, the focus lens isroughly brought into focus in advance based on the image pickup signalsobtained for live view before the still image is taken. When the nextsecond still image is taken, the direction to drive the focus lens isdetermined based on the difference between the contrast evaluation valuethat is based on the image pickup signals obtained for live view and thecontrast evaluation value that is based on the image pickup signalsobtained when the first still image is taken. The same holds true alsowhen the third and subsequent still images are taken.

Referring back to FIG. 29, the operation of the contrast evaluationvalue calculator 218 is described. After the image pickup signals areread out from all of the image pickup pixels (time TF4), the contrastevaluation value calculator 218 calculates the contrast evaluation valuebased on the image pickup signals in which high-frequency components areextracted (period between TF4 and TF6). The solid-state image pickupelement 102 outputs the calculated contrast evaluation value to thecontroller 106 via the signal processor 104 (period between TF6 andTF8). The contrast evaluation value is output in a form of being addedto the image pickup signals, and hence there is a blanking period fromwhen the image pickup signals are read out to when the contrastevaluation value is calculated (period between TF4 and TF6).

The controller 106 acquires the contrast evaluation value from thecontrast evaluation value calculator 218 (time TF8). Then, thecontroller 106 drives the focus lens of the optical mechanism 1011 basedon the contrast evaluation value, to thereby perform the in-focusoperation (period between TF8 and TF9). After the in-focus operation iscompleted, as in the case of the phase difference AF, the operation ofexposure to the pixels 201 is started to take the next still image (timeTF9).

As described above, in the sixth embodiment, after the phase differencesignals are read out, the image pickup signals are read out. In thismanner, the phase difference evaluation values can be calculated withoutwaiting for the completion of the readout of the image pickup signalsfrom all of the image pickup pixels. Therefore, the hybrid AF can bestarted at an earlier timing. As a result, for example, the still imagecontinuous shooting speed can be increased.

Further, in the sixth embodiment, whether or not to calculate thecontrast evaluation value is determined based on the phase differenceevaluation value. In this manner, when the focus to the object can beachieved by the focus detection using the phase difference detectionmethod, the focus detection using the contrast detection method isomitted, and hence the in-focus operation can be increased in speed.Further, even when it is difficult to achieve the focus to the object byonly the focus detection using the phase difference detection method,the focus detection using the contrast detection method can be startedas soon as the readout of the image pickup signals from all of the imagepickup pixels is completed. Therefore, the in-focus operation can stillbe increased in speed.

FIG. 30 is a flow chart for illustrating a method of controlling thesolid-state image pickup element 102 according to the sixth embodiment.The controller 106 of the sixth embodiment performs hybrid AF inaccordance with the flow chart of FIG. 30. For example, when the stillimage continuous shooting mode is started due to the operation performedby the user on the operation unit 108, the controller 106 controls theoptical mechanism 1011 to open the mechanical shutter (Step S3001). As aresult, the exposure to the pixels 201 is started (Step S3002). Then,after an elapse of a predetermined exposure period, the controller 106controls the optical mechanism 1011 to close the mechanical shutter(Step S3003).

Subsequently, the phase difference evaluation value calculator 217starts the readout of the phase difference signals in order to calculatethe phase difference evaluation value necessary for the phase differenceAF (Step S3004). Then, the phase difference evaluation value calculator217 repeats the readout of the phase difference signal until the readoutof the phase difference signals from all of the phase differencedetection pixels is completed (Step S3005). After the readout of thephase difference signals from all of the phase difference detectionpixels is completed, the readout of the image pickup signals from theimage pickup pixels is started (Step S3006). The read image pickupsignals are output to the contrast evaluation value calculator 218 andthe signal processor 104.

The phase difference evaluation value calculator 217 calculates thephase difference evaluation value based on the read phase differencesignals before the readout of the image pickup signals is completed(Step S3007). The determination unit of the solid-state image pickupelement 102 calculates the evaluation value reliability fnclvl, which isthe degree of coincidence of the pair of phase difference signals, basedon the phase difference evaluation value, to thereby determine whetheror not the evaluation value reliability fnclvl is equal to or largerthan a predetermined threshold value fth (Step S3008).

In Step S3008, when the reliability fnclvl is equal to or larger thanthe threshold value fth (YES), the determination unit of the solid-stateimage pickup element 102 determines that the focus to the object can beachieved by the focus detection using the phase difference detectionmethod, and thus the processing proceeds to Step S3009. Then, thecompletion of the readout of the image pickup signals from all of theimage pickup pixels is waited for.

After the image pickup signals are read out from all of the image pickuppixels, the solid-state image pickup element 102 outputs the phasedifference evaluation value to the controller 106 via the signalprocessor 104 in a form of being added to the image pickup signals (StepS3010). The controller 106 determines the feed-back control amount basedon the phase difference evaluation value, and controls the drive unit103 to drive the focus lens in the optical mechanism 1011, to therebyperform the in-focus operation (Step S3011).

On the other hand, in Step S3008, when the reliability fnclvl is smallerthan the threshold value fth (NO), the determination unit of thesolid-state image pickup element 102 determines that it is difficult toachieve the focus to the object by only the focus detection using thephase difference detection method, and the processing proceeds to StepS3012. Then, completion of the readout of the image pickup signals fromall of the image pickup pixels is waited for.

After the image pickup signals are read out from all of the image pickuppixels, the contrast evaluation value calculator 218 calculates thecontrast evaluation value based on the image pickup signals (StepS3013). The solid-state image pickup element 102 outputs the contrastevaluation value to the controller 106 via the signal processor 104 in aform of being added to the image pickup signals (Step S3014). Thecontroller 106 determines the feed-back control amount based on thephase difference evaluation value, and controls the drive unit 103 todrive the focus lens in the optical mechanism 1011, to thereby performthe in-focus operation (Step S3015).

After the in-focus operation is completed, the controller 106 verifieswhether or not the still image continuous shooting mode is ended due tothe operation performed by the user on the operation unit 108, forexample (Step S3016). When the still image continuous shooting mode isended (YES), the hybrid AF processing is ended. On the other hand, whenthe still image continuous shooting mode is continued (NO), theprocessing returns to Step S3001 so that the processing of theabove-mentioned flow chart is repeated to take the next still image.

As described above, the solid-state image pickup element according tothe sixth embodiment includes the phase difference evaluation valuecalculator that calculates the phase difference evaluation value basedon the phase difference signals, and the contrast evaluation valuecalculator that calculates the contrast evaluation value based on theimage pickup signals. Further, before the readout of the image pickupsignals is completed, the phase difference evaluation value iscalculated based on the phase difference signals, and whether or not tocalculate the contrast evaluation value is determined based on the phasedifference evaluation value.

With such a configuration, the phase difference evaluation value can becalculated without waiting for the completion of the readout of theimage pickup signals from all of the image pickup pixels, and hence thetiming to start the hybrid AF can be set earlier. That is, focusdetection using a combination of the phase difference detection methodand the contrast detection method can be performed at a higher speed.

Further, when the focus to the object can be achieved with the focusdetection using the phase difference detection method, the focusdetection using the contrast detection method can be omitted, and hencethe focus detection can be more increased in speed. The determinationunit that determines whether or not to calculate the contrast evaluationvalue may be the phase difference evaluation value calculator 217, ormay be the contrast evaluation value calculator 218. Alternatively, thedetermination unit may be a peripheral circuit (circuit) provided on thesecond semiconductor chip 21 illustrated in FIG. 20.

Seventh Embodiment

Next, an image pickup apparatus according to a seventh embodiment isdescribed with reference to FIG. 31 to FIG. 33. In the sixth embodiment,a configuration in which one column signal line 202 is provided for eachcolumn of the solid-state image pickup element 102 is described. Incontrast, in the seventh embodiment, description is given of aconfiguration in which at least two column signal lines 202 a and 202 bare provided for each column of a solid-state image pickup element 102 bso that the image pickup signals from the image pickup pixels and thephase difference signals from the phase difference detection pixels canbe read out in parallel.

FIG. 31 is a schematic diagram for illustrating a configuration of theimage pickup element 102 b according to the seventh embodiment. Thesolid-state image pickup element 102 b according to the seventhembodiment illustrated in FIG. 31 includes the two column signal lines202 a and 202 b for each column. Further, the solid-state image pickupelement 102 b includes, on one side of the pixel array of thesolid-state image pickup element 102 b, AD conversion units 211 a, thecolumn scanning circuit 213 a, and the horizontal signal line 215 a, andon the other side of the pixel array, AD conversion units 211 b, thecolumn scanning circuit 213 b, and the horizontal signal line 215 b.Other configurations are substantially the same as those of the sixthembodiment, and hence description thereof is omitted. Configurationsdifferent from those of the sixth embodiment are mainly described below.

On the first semiconductor chip 20 of the solid-state image pickupelement 102 b according to the seventh embodiment, similarly to thesixth embodiment, the plurality of pixels 201 are arranged in matrix.The column signal line 202 b is connected to, among those pixels 201,the pixels 201 in the phase difference detection row in which thesub-pixels SHA and SHB are also arranged. Meanwhile, the column signalline 202 a is connected to the pixel 201 in the image pickup row inwhich only the normal image pickup pixels are arranged. The presentinvention is not necessarily limited to such a configuration, and in theseventh embodiment, it is only required that all of the sub-pixels SHAand SHB in the pixel array be connected to the column signal lines 202b.

Meanwhile, on the second semiconductor chip 21 of the solid-state imagepickup element 102 b according to the seventh embodiment, the ADconversion units 211 a, the AD conversion units 211 b, the columnscanning circuit 213 a, and the column scanning circuit 213 b arearranged. Each of the AD conversion units 211 a is controlled by thecolumn scanning circuit 213 a to subject the image pickup signals outputto the column signal line 202 a to AD conversion and then output theimage pickup signals to the horizontal signal line 215 a. Similarly,each of the AD conversion units 211 b is controlled by the columnscanning circuit 213 b to subject the image pickup signals or the phasedifferent signals output to the column signal line 202 b to ADconversion and then output the signals to the horizontal signal line 215b. The column scanning circuits 213 a and 213 b are controlled by thetiming control circuit 214.

With such a configuration, the image pickup signal output from thenormal image pickup pixel is output to the contrast evaluation valuecalculator 218 and the signal processor 104 via the column signal line202 a or the column signal line 202 b. Meanwhile, the phase differencesignal output from the phase difference detection pixel including thesub-pixels SHA and SHB is output to the switch 216 via the column signalline 202 b. The switch 216 switches the output destination of the signalread out via the column signal line 202 b. With the switch 216, theimage pickup signal is output to the contrast evaluation valuecalculator 218 and the signal processor 104, and the phase differencesignal is output to the phase difference evaluation value calculator217.

Now, referring back to FIG. 28, the hybrid AF in the image pickupapparatus according to the seventh embodiment is described. The rowscanning circuit 212 of the seventh embodiment performs in parallel thescanning of the phase difference detection rows in which the sub-pixelsSHA and SHB are arranged and the scanning of the image pickup rows inwhich only the normal image pickup pixels are arranged.

Specifically, when the image pickup is started due to the operation onthe operation unit 108 or the like, the first row, the second row, theninth row, the tenth row, and so on, which are the phase differencedetection rows, are sequentially scanned. Simultaneously, the third toeighth rows, the eleventh to sixteenth rows, and so on, which are theimage pickup rows, are sequentially scanned. As described above, theimage pickup signals and the phase difference signals are read out inparallel with use of the two column signal lines 202 a and 202 b. Thus,the image pickup signals and the phase difference signals can be readout at a higher speed.

The pixel arrangement in the solid-state image pickup element 102 b isnot necessarily limited to the eight-row cycle as illustrated in FIG.28, and is not required to be cyclic. In the solid-state image pickupelement 102 b according to the seventh embodiment, at least one phasedifference detection row in which the sub-pixels SHA and SHB arearranged and at least one image pickup row in which the sub-pixels SHAand SHB are not arranged are only required to be provided.

Next, the hybrid AF in the image pickup apparatus according to theseventh embodiment is described. FIG. 32 is a timing chart forillustrating the method of controlling the solid-state image pickupelement 102 b according to the seventh embodiment. In the timing chartof the sixth embodiment of FIG. 29, the readout of the image pickupsignals is started after the readout of the phase difference signals iscompleted. In contrast, in the timing chart of the seventh embodiment ofFIG. 32, the readout of the image pickup signals is performed inparallel to the readout of the phase difference signals. Otherconfigurations are substantially the same as those of the sixthembodiment, and hence description thereof is omitted. Configurationsdifferent from those of the sixth embodiment are mainly described below.

When the still image continuous shooting mode is started due to theoperation performed by the user on the operation unit 108, for example,the controller 106 sets the still image continuous shooting controlsignal representing the still image continuous shooting mode to HIGH(time T0). In this manner, at the same time as when the opening of themechanical shutter of the optical mechanism 1011 is opened, the chargesaccumulated in the pixels 201 are reset, and the exposure to the pixels201 is started. Then, after elapse of an exposure period set so as tosatisfy a predetermined exposure condition, the mechanical shutter ofthe optical mechanism 1011 is closed to block the light reflected fromthe object. Thus, the exposure to the pixels 201 is ended (time TF0).

The AF evaluation value calculator 1021 first performs focus detectionwith use of the phase difference detection method. The phase differenceevaluation value calculator 217 of the AF evaluation value calculator1021 starts the readout of the phase difference signals from the phasedifference detection pixels via the horizontal signal line 215 b and theswitch 216 in order to calculate the phase difference evaluation valuenecessary for the phase difference AF (time TF0). Simultaneously, thereadout of the image pickup signals from the image pickup pixels via thehorizontal signal line 215 a is started (time TF0).

After the phase difference signals are read out from all of the phasedifference detection pixels (time TF1), the phase difference evaluationvalue calculator 217 calculates the phase difference evaluation valuebased on the phase difference signals (period between TF1 and TF2).After that, the phase difference evaluation value calculator 217calculates the evaluation value reliability for determining whether ornot focus to the object can be achieved based on the calculated phasedifference evaluation value (period between TF2 and TF3). The processingto be performed thereafter is the same as that in the timing chart ofthe sixth embodiment of FIG. 29, and hence description thereof isomitted.

FIG. 33 is a flow chart for illustrating the method of controlling theimage pickup element 102 b according to the seventh embodiment. Thecontroller 106 of the seventh embodiment performs the hybrid AF inaccordance with the flow chart of FIG. 33. In the flow chart of thesixth embodiment of FIG. 30, the readout of the image pickup signals isstarted after the readout of the phase difference signals is completed.In contrast, in the flow chart of the seventh embodiment of FIG. 33, thereadout of the image pickup signals is performed in parallel to thereadout of the phase difference signals. Other configurations aresubstantially the same as those of the sixth embodiment, and hencedescription thereof is omitted. Configurations different from those ofthe sixth embodiment are mainly described below.

When the still image continuous shooting mode is started due to theoperation performed by the user on the operation unit 108, for example,the controller 106 controls the optical mechanism 1011 to open themechanical shutter (Step S3001). As a result, the exposure to the pixels201 is started (Step S3002). Then, after an elapse of a predeterminedexposure period, the controller 106 controls the optical mechanism 1011to close the mechanical shutter (Step S3003).

Subsequently, the phase difference evaluation value calculator 217starts the readout of the phase difference signals in order to calculatethe phase difference evaluation value necessary for the phase differenceAF (Step S3004). Simultaneously, the readout of the image pickup signalsfrom the image pickup pixels is started via the horizontal signal line215 a (Step S3006). The read image pickup signals are output to thecontrast evaluation value calculator 218 and the signal processor 104.Then, until the readout of the phase difference signals from all of thephase difference detection pixels is completed, the readout of the phasedifference signals is repeated (Step S3005).

After the readout of the phase difference signals from all of the phasedifference detection pixels is completed, the phase differenceevaluation value calculator 217 calculates the phase differenceevaluation value based on the read phase difference signals before thereadout of the image pickup signals is completed (Step S3007). Theprocessing to be performed thereafter is the same as that in the flowchart of the sixth embodiment of FIG. 30, and hence description thereofis omitted.

As described above, the solid-state image pickup element according tothe seventh embodiment performs in parallel the scanning of the row inwhich the phase difference detection pixels are arranged and thescanning of the row in which the phase difference detection pixels arenot arranged. In this manner, the readout of the image pickup signalsfrom the image pickup pixels is completed earlier, and hence the hybridAF can be started earlier. That is, the focus detection using thecombination of the phase difference detection method and the contrastdetection method can be further increased in speed.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processor(CPU), micro processor (MPU)) and may include a network of separatecomputers or separate processors to read out and execute the computerexecutable instructions. The computer executable instructions may beprovided to the computer, for example, from a network or the storagemedium. The storage medium may include, for example, one or more of ahard disk, a random-access memory (RAM), a read only memory (ROM), astorage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

For example, parts of the above-mentioned embodiments may be combined asappropriate.

Further, in the above-mentioned embodiments, description is given of anexample of a case in which the image pickup apparatus 100 is a digitalcamera, but the image pickup apparatus 100 is not limited to a digitalcamera. For example, the image pickup apparatus 100 may be a digitalvideo camera, or a smart phone, which is an electronic device havingboth of a function of a mobile information terminal and a function of amobile phone. Further, the image pickup apparatus 100 may be, forexample, a tablet terminal or a personal digital assistant (PDA).

Further, in the above-mentioned embodiments, description is given of anexample of a case in which the autofocus operation is performed whilethe live view display is performed, but the present invention is notlimited thereto. The present invention is applicable also to a case inwhich the autofocus operation is performed while a moving image istaken, for example.

Further, in the above-mentioned embodiments, description is given of anexample of a case in which the AF evaluation value is output from thesolid-state image pickup element 102 to the controller 106 so that thecontroller 106 may cause the drive unit 103 to control the opticalmechanism 1011, but the present invention is not limited thereto. Forexample, the AF evaluation value may be output from the solid-stateimage pickup element 102 directly to a functional block that controlsthe autofocus operation.

This application claims the benefit of Japanese Patent Application No.2016-206710, filed Oct. 21, 2016, No. 2016-206720, filed Oct. 21, 2016,and No. 2016-217766, filed Nov. 8, 2016 which are hereby incorporated byreference herein in their entirety.

What is claimed is:
 1. A solid-state image pickup element, comprising: apixel array including a plurality of pixels; a first calculator thatcalculates a phase difference evaluation value for focus detection by aphase difference detection method based on signal from the pixel; and asecond calculator that calculates a contrast evaluation value for focusdetection by a contrast detection method based on signal from the pixel,wherein, when the first calculator completes calculation of the phasedifference evaluation value, the phase difference evaluation value isoutput regardless of whether or not output of an image signal acquiredby the pixel array is completed, and wherein, when the second calculatorcompletes calculation of the contrast evaluation value, the contrastevaluation value is output regardless of whether or not output of theimage signal acquired by the pixel array is completed.
 2. A solid-stateimage pickup element according to claim 1, wherein the pixel array isformed on a first semiconductor chip, wherein the first calculator andthe second calculator are formed on a second semiconductor chipdifferent from the first semiconductor chip, and wherein the firstsemiconductor chip is stacked on the second semiconductor chip.
 3. Asolid-state image pickup element according to claim 1, wherein each ofthe phase difference evaluation value and the contrast evaluation valueis output via an output terminal that is different from an outputterminal from which the image signal is output.
 4. A solid-state imagepickup element according to claim 1, wherein the phase differenceevaluation value, the contrast evaluation value, and the image signalare output via a common output terminal.
 5. A solid-state image pickupelement according to claim 4, wherein the phase difference evaluationvalue and the contrast evaluation value are inserted to blanking-periodparts of the image signal.
 6. A solid-state image pickup elementaccording to claim 1, wherein readout of the signal from the pixel forcalculating the phase difference evaluation value and readout of thesignal from the pixel for calculating the contrast evaluation value areperformed in parallel to readout of the image signal from the pixelarray for outputting the image signal.
 7. A solid-state image pickupelement according to claim 1, wherein the plurality of pixels include: afirst pixel that acquires a signal corresponding to a light flux passingthrough a first pupil region of an exit pupil of an image pickup opticalsystem; and a second pixel that acquires a signal corresponding to alight flux passing through a second pupil region different from thefirst pupil region of the exit pupil, and wherein the first calculatorcalculates the phase difference evaluation value based on the signalfrom the first pixel and the signal from the second pixel.
 8. A methodof controlling a solid-state image pickup element, comprising:outputting an image signal acquired by a pixel array including aplurality of pixels; calculating a phase difference evaluation value forfocus detection by a phase difference detection method based on signalfrom the pixel; calculating a contrast evaluation value for focusdetection by a contrast detection method based on signal from the pixel;outputting, when calculation of the phase difference evaluation value iscompleted, the phase difference evaluation value regardless of whetheror not output of the image signal acquired by the pixel array iscompleted; and outputting, when calculation of the contrast evaluationvalue is completed, the contrast evaluation value regardless of whetheror not output of the image signal acquired by the pixel array iscompleted.
 9. An image pickup apparatus, comprising: a solid-state imagepickup element, including: a pixel array including a plurality ofpixels; a first calculator that calculates a phase difference evaluationvalue for focus detection by a phase difference detection method basedon signal from the pixel; and a second calculator that calculates acontrast evaluation value for focus detection by a contrast detectionmethod based on signal from the pixel, the solid-state image pickupelement being configured to: output, when the first calculator completescalculation of the phase difference evaluation value, the phasedifference evaluation value regardless of whether or not output of animage signal acquired by the pixel array is completed; and output, whenthe second calculator completes calculation of the contrast evaluationvalue, the contrast evaluation value regardless of whether or not outputof the image signal acquired by the pixel array is completed; and acontroller that performs control for driving a focus lens based on thephase difference evaluation value and the contrast evaluation value. 10.A solid-state image pickup element, comprising: a pixel array includinga plurality of pixels arranged in matrix; a first calculator thatcalculates a phase difference evaluation value for focus detection by aphase difference detection method based on a signal from a pixel forphase difference detection included in the plurality of pixels; aninterpolation processor that generates a signal for compensating for adefect, which is caused in an image signal acquired by the pixel arrayand caused because the pixel for phase difference detection is includedin the plurality of pixels, through interpolation using a signal from apixel other than the pixel for phase difference detection; and a secondcalculator that calculates a contrast evaluation value for focusdetection by a contrast detection method based on the image signalincluding the signal generated by the interpolation processor throughthe interpolation.
 11. A solid-state image pickup element according toclaim 10, wherein, when the first calculator completes calculation ofthe phase difference evaluation value, the phase difference evaluationvalue is output regardless of whether or not output of the image signalacquired by the pixel array is completed.
 12. A solid-state image pickupelement according to claim 10, wherein the pixel array is formed on afirst semiconductor chip, wherein the first calculator, theinterpolation processor, and the second calculator are formed on asecond semiconductor chip different from the first semiconductor chip,and wherein the first semiconductor chip is stacked on the secondsemiconductor chip.
 13. A solid-state image pickup element according toclaim 10, wherein the image signal subjected to the interpolation by theinterpolation processor is output.
 14. A solid-state image pickupelement according to claim 10, wherein each of the phase differenceevaluation value and the contrast evaluation value is output via anoutput terminal that is different from an output terminal from which theimage signal subjected to the interpolation by the interpolationprocessor is output.
 15. A solid-state image pickup element according toclaim 10, wherein readout of a signal from a row including the pixel forphase difference detection is performed in parallel to readout of asignal from a row not including the pixel for phase differencedetection.
 16. A solid-state image pickup element according to claim 10,wherein the pixel array includes a plurality of pixels for phasedifference detection, wherein the plurality of pixels for phasedifference detection include: a first pixel for phase differencedetection, which acquires a signal corresponding to a light flux passingthrough a first pupil region of an exit pupil of an image pickup opticalsystem; and a second pixel for phase difference detection, whichacquires a signal corresponding to a light flux passing through a secondpupil region different from the first pupil region of the exit pupil,and wherein the first calculator calculates the phase differenceevaluation value based on the signal from the first pixel for phasedifference detection and the signal from the second pixel for phasedifference detection.
 17. A method of controlling a solid-state imagepickup element, comprising: calculating a phase difference evaluationvalue for focus detection by a phase difference detection method basedon a signal from a pixel for phase difference detection included in aplurality of pixels arranged in matrix in a pixel array; generating asignal for compensating for a defect, which is caused in an image signalacquired by the pixel array and caused because the pixel for phasedifference detection is included in the plurality of pixels, throughinterpolation using a signal from a pixel other than the pixel for phasedifference detection; and calculating a contrast evaluation value forfocus detection by a contrast detection method based on the image signalincluding the signal generated through the interpolation.
 18. An imagepickup apparatus, comprising: a solid-state image pickup elementincluding: a pixel array including a plurality of pixels arranged inmatrix; a first calculator that calculates a phase difference evaluationvalue for focus detection by a phase difference detection method basedon a signal from a pixel for phase difference detection included in theplurality of pixels; an interpolation processor that generates a signalfor compensating for a defect, which is caused in an image signalacquired by the pixel array and caused because the pixel for phasedifference detection is included in the plurality of pixels, throughinterpolation using a signal from a pixel other than the pixel for phasedifference detection; and a second calculator that calculates a contrastevaluation value for focus detection by a contrast detection methodbased on the image signal including the signal generated by theinterpolation processor through the interpolation; and a controller thatperforms control for driving a focus lens based on the phase differenceevaluation value and the contrast evaluation value.
 19. A solid-stateimage pickup element, comprising: a plurality of image pickup pixelsthat each receive light condensed by an image pickup optical system; aplurality of phase difference detection pixels that each receive a pairof light fluxes passing through different pupil partial regions of theimage pickup optical system; a circuit that reads out image pickupsignal from the image pickup pixel, and reads out phase differencesignal from the phase difference detection pixel; a contrast evaluationvalue calculator that calculates a contrast evaluation value based onthe image pickup signal; a phase difference evaluation value calculatorthat calculates a phase difference evaluation value based on the phasedifference signal before readout of the image pickup signal iscompleted; and a determination unit that determines whether or not tocalculate the contrast evaluation value based on the phase differenceevaluation value.
 20. A solid-state image pickup element according toclaim 19, wherein the contrast evaluation value is calculated when anevaluation value reliability representing a degree of coincidence of apair of the phase difference signals is smaller than a predeterminedthreshold value, and the contrast evaluation value is prevented frombeing calculated when the evaluation value reliability is equal to orlarger than the threshold value.
 21. A solid-state image pickup elementaccording to claim 19, wherein the plurality of image pickup pixels andthe plurality of phase difference detection pixels are arranged inmatrix, and wherein the circuit scans a row in which the phasedifference detection pixel is not arranged after scanning a row in whichthe phase difference detection pixel is arranged.
 22. A solid-stateimage pickup element according to claim 19, wherein the plurality ofimage pickup pixels and the plurality of phase difference detectionpixels are arranged in matrix, and wherein the circuit scans in parallela row in which the phase difference detection pixel is arranged and arow in which the phase difference detection pixel is not arranged.
 23. Asolid-state image pickup element according to claim 21, wherein thephase difference evaluation value calculator calculates the phasedifference evaluation value during a period in which the row in whichthe phase difference detection pixel is not arranged is scanned.
 24. Asolid-state image pickup element according to claim 21, furthercomprising: a column signal line arranged for each column; and a signalswitch that switches an output destination of a signal read out via thecolumn signal line, the signal switch outputs the image pickup signalsto the contrast evaluation value calculator and to output the phasedifference signals to the phase difference evaluation value calculator.25. A solid-state image pickup element according to claim 22, furthercomprising: a first column signal line arranged for each column; and asecond column signal line arranged for each column, wherein the imagepickup signals are output to the contrast evaluation value calculatorvia one of the first column signal line and the second column signalline, and the phase difference signals are output to the phasedifference evaluation value calculator via the second column signalline.
 26. A solid-state image pickup element according to claim 19,wherein the contrast evaluation value is output to a controller outsideof the solid-state image pickup element when the contrast evaluationvalue is calculated, and the phase difference evaluation value is outputto the controller when the contrast evaluation value is not calculated.27. A solid-state image pickup element according to claim 26, whereinone of the phase difference evaluation value and the contrast evaluationvalue is output to the controller in a form of being added to the imagepickup signals.
 28. An image pickup apparatus, comprising: a solid-stateimage pickup element including: a plurality of image pickup pixels thateach receive light condensed by an image pickup optical system; aplurality of phase difference detection pixels that each receive a pairof light fluxes passing through different pupil partial regions of theimage pickup optical system; a circuit that reads out image pickupsignal from the image pickup pixel, and to read out phase differencesignal from the phase difference detection pixel; a contrast evaluationvalue calculator that calculates a contrast evaluation value based onthe image pickup signal; a phase difference evaluation value calculatorthat calculates a phase difference evaluation value based on the phasedifference signal before readout of the image pickup signal iscompleted; and a determination unit that determines whether or not tocalculate the contrast evaluation value based on the phase differenceevaluation value; and a controller that performs an in-focus operationof the image pickup optical system based on any one of the phasedifference evaluation value and the contrast evaluation value.
 29. Amethod of controlling a solid-state image pickup element, thesolid-state image pickup element including: a plurality of image pickuppixels that each receive light condensed by an image pickup opticalsystem; a plurality of phase difference detection pixels that eachreceive a pair of light fluxes passing through different pupil partialregions of the image pickup optical system; and a circuit that reads outimage pickup signal from the image pickup pixel, and to read out phasedifference signal from the phase difference detection pixel, the methodcomprising: calculating a phase difference evaluation value based on thephase difference signal before readout of the image pickup signal iscompleted; and determining whether or not to calculate a contrastevaluation value based on the image pickup signal, depending on thephase difference evaluation value.